From sandro.tosi at gmail.com  Wed Jun  1 00:59:08 2011
From: sandro.tosi at gmail.com (Sandro Tosi)
Date: Wed, 1 Jun 2011 00:59:08 +0200
Subject: [Python-Dev] Some additions to .hgignore
Message-ID: <BANLkTinspoCeN5r4S3LW9=87NF6wEAkFRA@mail.gmail.com>

Hi all,
following http://docs.python.org/devguide/coverage.html doc you'll end
up with several "new" files/dirs in your checkout:

- .coverage, used by coveragepy to save its info
- coverage/ , the symlink to coveragepy/coverage
- htmlcov/ , the dir where the coverage HTML pages are written

I think they should be added to .hgignore so that hg st won't show them.

I'm writing here since I don't think an issue is needed for such
matter, if that's not the case, I apologize.

Regards,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

From martin at v.loewis.de  Wed Jun  1 07:37:09 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 01 Jun 2011 07:37:09 +0200
Subject: [Python-Dev] Sniffing passwords from PyPI using
	insecure	connection
In-Reply-To: <is3e5o$qou$1@dough.gmane.org>
References: <BANLkTikfTXNrvBQpJ-_kXHePon7ynAmOGw@mail.gmail.com>
	<is3e5o$qou$1@dough.gmane.org>
Message-ID: <4DE5D005.5000001@v.loewis.de>

> The requested one character change is
> -    DEFAULT_REPOSITORY = 'http://pypi.python.org/pypi'
> +    DEFAULT_REPOSITORY = 'https://pypi.python.org/pypi'
> 
> If Tarek (or perhaps Eric) agree that it is appropriate and otherwise
> innocuous, then Martin and Barry can decide whether to include in 2.5/2.6.

I don't plan any further 2.5 releases, so unless a critical security
issue pops up, 2.5.6 will have been the last release.

Regards,
Martin

From tjreedy at udel.edu  Wed Jun  1 08:33:53 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 01 Jun 2011 02:33:53 -0400
Subject: [Python-Dev] Sniffing passwords from PyPI using insecure
	connection
In-Reply-To: <4DE5D005.5000001@v.loewis.de>
References: <BANLkTikfTXNrvBQpJ-_kXHePon7ynAmOGw@mail.gmail.com>	<is3e5o$qou$1@dough.gmane.org>
	<4DE5D005.5000001@v.loewis.de>
Message-ID: <is4mgf$6uh$1@dough.gmane.org>

On 6/1/2011 1:37 AM, "Martin v. L?wis" wrote:
>> The requested one character change is
>> -    DEFAULT_REPOSITORY = 'http://pypi.python.org/pypi'
>> +    DEFAULT_REPOSITORY = 'https://pypi.python.org/pypi'
>>
>> If Tarek (or perhaps Eric) agree that it is appropriate and otherwise
>> innocuous, then Martin and Barry can decide whether to include in 2.5/2.6.
>
> I don't plan any further 2.5 releases, so unless a critical security
> issue pops up, 2.5.6 will have been the last release.

OK. I removed 2.5 from all open issues, closing a few. You could remove 
2.5 from the displayed version list so that people cannot add it back or 
to new issues.

-- 
Terry Jan Reedy



From martin at v.loewis.de  Wed Jun  1 08:48:04 2011
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 01 Jun 2011 08:48:04 +0200
Subject: [Python-Dev] Extending os.chown() to accept user/group names
In-Reply-To: <1306343861.20117.4.camel@marge>
References: <BANLkTikXottB8xfXbe1y48P-GWVxhKn2=Q@mail.gmail.com>	<BANLkTi=8K1=ZQfxAD0nK0wAaCy33FRmEEw@mail.gmail.com>
	<1306343861.20117.4.camel@marge>
Message-ID: <4DE5E0A4.8050608@v.loewis.de>

> Le mercredi 25 mai 2011 ? 18:46 +0200, Charles-Fran?ois Natali a ?crit :
>> While we're at it, adding a "recursive" argument to this shutil.chown
>> could also be useful.
> 
> I don't like the idea of a recursive flag.

I think shutil.chown should aim to mimic chown(1). At least GNU chown
has a -R flag (not sure about POSIX chown), and it's useful in practice.

Regards,
Martin

From ezio.melotti at gmail.com  Wed Jun  1 12:37:09 2011
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Wed, 01 Jun 2011 13:37:09 +0300
Subject: [Python-Dev] [Python-checkins] cpython: test.support: add
 requires_mac_ver() function
In-Reply-To: <E1QRifG-0004qn-Uc@dinsdale.python.org>
References: <E1QRifG-0004qn-Uc@dinsdale.python.org>
Message-ID: <4DE61655.5000401@gmail.com>

Hi,

On 01/06/2011 13.28, victor.stinner wrote:
> http://hg.python.org/cpython/rev/35212b113730
> changeset:   70574:35212b113730
> user:        Victor Stinner<victor.stinner at haypocalc.com>
> date:        Wed Jun 01 12:28:04 2011 +0200
> summary:
>    test.support: add requires_mac_ver() function

I would expect this to be a decorator, similar to requires_IEEE_754 and 
requires_zlib.

> Add also linux_version() to __all__.

For consistency, this should be a decorator too, possibly named 
requires_linux_ver().
A requires_windows(_ver) sounds like a useful addition too, given the 
number of tests that are specific to Windows.

> files:
>    Lib/test/support.py   |  22 +++++++++++++++++++++-
>    Lib/test/test_math.py |   7 ++++---
>    2 files changed, 25 insertions(+), 4 deletions(-)
>
>
> diff --git a/Lib/test/support.py b/Lib/test/support.py
> --- a/Lib/test/support.py
> +++ b/Lib/test/support.py
> @@ -37,7 +37,8 @@
>       "Error", "TestFailed", "ResourceDenied", "import_module",
>       "verbose", "use_resources", "max_memuse", "record_original_stdout",
>       "get_original_stdout", "unload", "unlink", "rmtree", "forget",
> -    "is_resource_enabled", "requires", "find_unused_port", "bind_port",
> +    "is_resource_enabled", "requires", "linux_version", "requires_mac_ver",
> +    "find_unused_port", "bind_port",
>       "IPV6_ENABLED", "is_jython", "TESTFN", "HOST", "SAVEDCWD", "temp_cwd",
>       "findfile", "sortdict", "check_syntax_error", "open_urlresource",
>       "check_warnings", "CleanImport", "EnvironmentVarGuard", "TransientResource",
> @@ -299,6 +300,25 @@
>       except ValueError:
>           return 0, 0, 0
>
> +def requires_mac_ver(*min_version):
> +    """Raise SkipTest if the OS is Mac OS X and the OS X version if less than
> +    min_version.
> +
> +    For example, support.requires_linux_version(10, 5) raises SkipTest if the

This should be requires_mac_ver

> +    version is less than 10.5.
> +    """
> +    if sys.platform != 'darwin':
> +        return
> +    version_txt = platform.mac_ver()[0]
> +    try:
> +        version = tuple(map(int, version_txt.split('.')))
> +    except ValueError:
> +        return
> +    if version<  min_version:
> +        min_version_txt = '.'.join(map(str, min_version))
> +        raise unittest.SkipTest("Mac OS X %s or higher required, not %s"
> +                                % (min_version_txt, version_txt))
> +
>   HOST = 'localhost'
>
>   def find_unused_port(family=socket.AF_INET, socktype=socket.SOCK_STREAM):
> diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py
> --- a/Lib/test/test_math.py
> +++ b/Lib/test/test_math.py
> @@ -2,6 +2,7 @@
>   # XXXX Should not do tests around zero only
>
>   from test.support import run_unittest, verbose, requires_IEEE_754
> +from test import support
>   import unittest
>   import math
>   import os
> @@ -669,10 +670,10 @@
>           self.assertTrue(math.isnan(math.log2(NAN)))
>
>       @requires_IEEE_754
> -    @unittest.skipIf(sys.platform == 'darwin'
> -                     and platform.mac_ver()[0].startswith('10.4.'),
> -                     'Mac OS X Tiger log2() is not accurate enough')
>       def testLog2Exact(self):
> +        # log2() is not accurate enough on Mac OS X Tiger (10.4)
> +        support.requires_mac_ver(10, 5)
> +
>           # Check that we get exact equality for log2 of powers of 2.
>           actual = [math.log2(math.ldexp(1.0, n)) for n in range(-1074, 1024)]
>           expected = [float(n) for n in range(-1074, 1024)]
>

Best Regards,
Ezio Melotti

From fuzzyman at voidspace.org.uk  Wed Jun  1 12:49:32 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 01 Jun 2011 11:49:32 +0100
Subject: [Python-Dev] Some additions to .hgignore
In-Reply-To: <BANLkTinspoCeN5r4S3LW9=87NF6wEAkFRA@mail.gmail.com>
References: <BANLkTinspoCeN5r4S3LW9=87NF6wEAkFRA@mail.gmail.com>
Message-ID: <4DE6193C.3000708@voidspace.org.uk>

On 31/05/2011 23:59, Sandro Tosi wrote:
> Hi all,
> following http://docs.python.org/devguide/coverage.html doc you'll end
> up with several "new" files/dirs in your checkout:
>
> - .coverage, used by coveragepy to save its info
> - coverage/ , the symlink to coveragepy/coverage
> - htmlcov/ , the dir where the coverage HTML pages are written
>
> I think they should be added to .hgignore so that hg st won't show them.
>
> I'm writing here since I don't think an issue is needed for such
> matter, if that's not the case, I apologize.

That sounds good to me. An issue certainly wouldn't hurt.

All the best,

Michael

> Regards,


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From barry at python.org  Wed Jun  1 13:08:18 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 1 Jun 2011 07:08:18 -0400
Subject: [Python-Dev] Sniffing passwords from PyPI using insecure
 connection
In-Reply-To: <is4mgf$6uh$1@dough.gmane.org>
References: <BANLkTikfTXNrvBQpJ-_kXHePon7ynAmOGw@mail.gmail.com>
	<is3e5o$qou$1@dough.gmane.org> <4DE5D005.5000001@v.loewis.de>
	<is4mgf$6uh$1@dough.gmane.org>
Message-ID: <20110601070818.244edc03@neurotica.wooz.org>

On Jun 01, 2011, at 02:33 AM, Terry Reedy wrote:

>On 6/1/2011 1:37 AM, "Martin v. L?wis" wrote:
>>> The requested one character change is
>>> -    DEFAULT_REPOSITORY = 'http://pypi.python.org/pypi'
>>> +    DEFAULT_REPOSITORY = 'https://pypi.python.org/pypi'
>>>
>>> If Tarek (or perhaps Eric) agree that it is appropriate and otherwise
>>> innocuous, then Martin and Barry can decide whether to include in 2.5/2.
>6.
>>
>> I don't plan any further 2.5 releases, so unless a critical security
>> issue pops up, 2.5.6 will have been the last release.
>
>OK. I removed 2.5 from all open issues, closing a few. You could remove 2.5
>from the displayed version list so that people cannot add it back or to new
>issues.

I followed up on the tracker.  I'm +0 on adding this to 2.6, but not until
after the 2.6.7 release on Friday.

How well has this change been tested?  Are there people for whom this could
break things?

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110601/0c40d655/attachment.pgp>

From lukasz at langa.pl  Wed Jun  1 16:35:16 2011
From: lukasz at langa.pl (=?iso-8859-2?Q?=A3ukasz_Langa?=)
Date: Wed, 1 Jun 2011 16:35:16 +0200
Subject: [Python-Dev] Extending os.chown() to accept user/group names
In-Reply-To: <4DE5E0A4.8050608@v.loewis.de>
References: <BANLkTikXottB8xfXbe1y48P-GWVxhKn2=Q@mail.gmail.com>	<BANLkTi=8K1=ZQfxAD0nK0wAaCy33FRmEEw@mail.gmail.com>
	<1306343861.20117.4.camel@marge> <4DE5E0A4.8050608@v.loewis.de>
Message-ID: <3DB8721D-E2F6-407E-B141-8C432AB83DA9@langa.pl>


Wiadomo?? napisana przez Martin v. L?wis w dniu 2011-06-01, o godz. 08:48:

>> Le mercredi 25 mai 2011 ? 18:46 +0200, Charles-Fran?ois Natali a ?crit :
>>> While we're at it, adding a "recursive" argument to this shutil.chown
>>> could also be useful.
>> 
>> I don't like the idea of a recursive flag.
> 
> I think shutil.chown should aim to mimic chown(1). At least GNU chown
> has a -R flag (not sure about POSIX chown), and it's useful in practice.

cp(1) and rm(1) also have "-r", still there are `shutil.copytree` and `shutil.rmtree`. I would like to keep that consistency, e.g. to have `shutil.chown` and `shutil.chowntree` as previously discussed by Charles-Fran?ois, Nick and Petri.

-- 
Best regards,
?ukasz Langa
Senior Systems Architecture Engineer

IT Infrastructure Department
Grupa Allegro Sp. z o.o.

From merwok at netwok.org  Wed Jun  1 19:15:54 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 01 Jun 2011 19:15:54 +0200
Subject: [Python-Dev] Docs for the packaging module
In-Reply-To: <b35dbbc50253fd96327a62e906b8259f@netwok.org>
References: <b35dbbc50253fd96327a62e906b8259f@netwok.org>
Message-ID: <4DE673CA.6070503@netwok.org>

Hi,

Looks like nobody cares enough about the packaging docs :)  If there is
no feedback, here?s what I propose to do:

- Add new Doc/library/packaging* files (library reference for developers
of packaging tools)

- Add new packaging-based ?Installing Python Projects? to Doc/install,
replacing old distutils docs

- Add new packaging-based ?Distributing Python Projects? Doc/packaging,
replacing old distutils docs (+ link to it from the main HTML page
instead of linking to Doc/distutils)

For users needing the legacy distutils docs in 3.3, I would move the
older distutils Doc/install/index.rst to Doc/distutils/install.rst, and
add a link to Doc/distutils in Doc/library/distutils (because the main
page would no longer link to Doc/distutils).

Regards

From benjamin at python.org  Wed Jun  1 19:37:35 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 1 Jun 2011 12:37:35 -0500
Subject: [Python-Dev] Docs for the packaging module
In-Reply-To: <4DE673CA.6070503@netwok.org>
References: <b35dbbc50253fd96327a62e906b8259f@netwok.org>
	<4DE673CA.6070503@netwok.org>
Message-ID: <BANLkTikEGtBriAypECO89OH4Vk46j0DZ3Q@mail.gmail.com>

2011/6/1 ?ric Araujo <merwok at netwok.org>:
> Hi,
>
> Looks like nobody cares enough about the packaging docs :)

Perhaps your solutions are perfect already. :)

>?If there is
> no feedback, here?s what I propose to do:
>
> - Add new Doc/library/packaging* files (library reference for developers
> of packaging tools)
>
> - Add new packaging-based ?Installing Python Projects? to Doc/install,
> replacing old distutils docs
>
> - Add new packaging-based ?Distributing Python Projects? Doc/packaging,
> replacing old distutils docs (+ link to it from the main HTML page
> instead of linking to Doc/distutils)
>
> For users needing the legacy distutils docs in 3.3, I would move the
> older distutils Doc/install/index.rst to Doc/distutils/install.rst, and
> add a link to Doc/distutils in Doc/library/distutils (because the main
> page would no longer link to Doc/distutils).

Or we could just delete them and have people look at docs for old
versions of Python.



-- 
Regards,
Benjamin

From merwok at netwok.org  Wed Jun  1 19:46:43 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 01 Jun 2011 19:46:43 +0200
Subject: [Python-Dev] Docs for the packaging module
In-Reply-To: <BANLkTikEGtBriAypECO89OH4Vk46j0DZ3Q@mail.gmail.com>
References: <b35dbbc50253fd96327a62e906b8259f@netwok.org>	<4DE673CA.6070503@netwok.org>
	<BANLkTikEGtBriAypECO89OH4Vk46j0DZ3Q@mail.gmail.com>
Message-ID: <4DE67B03.5000008@netwok.org>

Hi Benjamin,

>> For users needing the legacy distutils docs in 3.3, I would move the
>> older distutils Doc/install/index.rst to Doc/distutils/install.rst, and
>> add a link to Doc/distutils in Doc/library/distutils (because the main
>> page would no longer link to Doc/distutils).
> 
> Or we could just delete them and have people look at docs for old
> versions of Python.

Given that we haven?t removed the code, I would prefer to keep the doc too.

Tarek has given me the greenlight, so I will commit the new docs
momentarily.

Cheers

From ziade.tarek at gmail.com  Wed Jun  1 19:58:58 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 1 Jun 2011 19:58:58 +0200
Subject: [Python-Dev] Docs for the packaging module
In-Reply-To: <4DE673CA.6070503@netwok.org>
References: <b35dbbc50253fd96327a62e906b8259f@netwok.org>
	<4DE673CA.6070503@netwok.org>
Message-ID: <BANLkTi=c-vEa-i8GoDuZnA2bXAsMRNUrGw@mail.gmail.com>

I do care :)

Looks fine

Please push, as we can always change and fix things afterwards in the tip
before 3.3 is out.
Le 1 juin 2011 19:38, "?ric Araujo" <merwok at netwok.org> a ?crit :
> Hi,
>
> Looks like nobody cares enough about the packaging docs :) If there is
> no feedback, here?s what I propose to do:
>
> - Add new Doc/library/packaging* files (library reference for developers
> of packaging tools)
>
> - Add new packaging-based ?Installing Python Projects? to Doc/install,
> replacing old distutils docs
>
> - Add new packaging-based ?Distributing Python Projects? Doc/packaging,
> replacing old distutils docs (+ link to it from the main HTML page
> instead of linking to Doc/distutils)
>
> For users needing the legacy distutils docs in 3.3, I would move the
> older distutils Doc/install/index.rst to Doc/distutils/install.rst, and
> add a link to Doc/distutils in Doc/library/distutils (because the main
> page would no longer link to Doc/distutils).
>
> Regards
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110601/26dabc9a/attachment.html>

From rdmurray at bitdance.com  Wed Jun  1 19:58:59 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 01 Jun 2011 13:58:59 -0400
Subject: [Python-Dev] Docs for the packaging module
In-Reply-To: <4DE673CA.6070503@netwok.org>
References: <b35dbbc50253fd96327a62e906b8259f@netwok.org>
	<4DE673CA.6070503@netwok.org>
Message-ID: <20110601175859.D425A25059F@webabinitio.net>

On Wed, 01 Jun 2011 19:15:54 +0200, <merwok at netwok.org> wrote:
> Looks like nobody cares enough about the packaging docs :)  If there is
> no feedback, here???s what I propose to do:

I think you should go ahead and make your changes, and then we'll
be able to see what it really looks like and decide if anything
ought to be tweaked.

--
R. David Murray           http://www.bitdance.com

From tdkadich at gmail.com  Wed Jun  1 19:54:13 2011
From: tdkadich at gmail.com (Timothy Kadich)
Date: Wed, 1 Jun 2011 10:54:13 -0700
Subject: [Python-Dev] Question about Module Loading
Message-ID: <BANLkTinkvH8P6Ojfk2uBf5R1GtGH_zMfhQ@mail.gmail.com>

If a Python program imported a module, say numpy for example, where in the
source is this line interpreted and then handled by import.c ?

Thank you in advance!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110601/56533e3e/attachment.html>

From rdmurray at bitdance.com  Wed Jun  1 21:03:54 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 01 Jun 2011 15:03:54 -0400
Subject: [Python-Dev] Question about Module Loading
In-Reply-To: <BANLkTinkvH8P6Ojfk2uBf5R1GtGH_zMfhQ@mail.gmail.com>
References: <BANLkTinkvH8P6Ojfk2uBf5R1GtGH_zMfhQ@mail.gmail.com>
Message-ID: <20110601190355.7D21225059F@webabinitio.net>

On Wed, 01 Jun 2011 10:54:13 -0700, Timothy Kadich <tdkadich at gmail.com> wrote:
> If a Python program imported a module, say numpy for example, where in the
> source is this line interpreted and then handled by import.c ?

Your question is not as simple as you think (I think), but I'm
guessing you will want to look at Python/ceval.c.

--
R. David Murray           http://www.bitdance.com

From benjamin at python.org  Wed Jun  1 21:04:49 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 1 Jun 2011 14:04:49 -0500
Subject: [Python-Dev] Question about Module Loading
In-Reply-To: <BANLkTinkvH8P6Ojfk2uBf5R1GtGH_zMfhQ@mail.gmail.com>
References: <BANLkTinkvH8P6Ojfk2uBf5R1GtGH_zMfhQ@mail.gmail.com>
Message-ID: <BANLkTinYTvbQzrwGm1Q==rUoBPPu_UUFgQ@mail.gmail.com>

2011/6/1 Timothy Kadich <tdkadich at gmail.com>:
> If a Python program imported a module, say numpy for example, where in the
> source is this line interpreted and then handled by import.c ?

Many different files. Start from TARGET(IMPORT_NAME) in ceval.c.


-- 
Regards,
Benjamin

From tdkadich at gmail.com  Thu Jun  2 00:43:08 2011
From: tdkadich at gmail.com (Timothy Kadich)
Date: Wed, 1 Jun 2011 15:43:08 -0700
Subject: [Python-Dev] Question about Module Loading
In-Reply-To: <BANLkTinYTvbQzrwGm1Q==rUoBPPu_UUFgQ@mail.gmail.com>
References: <BANLkTinkvH8P6Ojfk2uBf5R1GtGH_zMfhQ@mail.gmail.com>
	<BANLkTinYTvbQzrwGm1Q==rUoBPPu_UUFgQ@mail.gmail.com>
Message-ID: <BANLkTimzw5cLRo6vGuHLR76prpc_eJNBCQ@mail.gmail.com>

I don't understand what you mean by "TARGET(IMPORT_NAME)". I can't find that
string in ceval.c.

On 1 June 2011 12:04, Benjamin Peterson <benjamin at python.org> wrote:

> 2011/6/1 Timothy Kadich <tdkadich at gmail.com>:
> > If a Python program imported a module, say numpy for example, where in
> the
> > source is this line interpreted and then handled by import.c ?
>
> Many different files. Start from TARGET(IMPORT_NAME) in ceval.c.
>
>
> --
> Regards,
> Benjamin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110601/d0cbfa4c/attachment.html>

From tdkadich at gmail.com  Thu Jun  2 00:51:10 2011
From: tdkadich at gmail.com (Timothy Kadich)
Date: Wed, 1 Jun 2011 15:51:10 -0700
Subject: [Python-Dev] Question about Module Loading
In-Reply-To: <BANLkTimzw5cLRo6vGuHLR76prpc_eJNBCQ@mail.gmail.com>
References: <BANLkTinkvH8P6Ojfk2uBf5R1GtGH_zMfhQ@mail.gmail.com>
	<BANLkTinYTvbQzrwGm1Q==rUoBPPu_UUFgQ@mail.gmail.com>
	<BANLkTimzw5cLRo6vGuHLR76prpc_eJNBCQ@mail.gmail.com>
Message-ID: <BANLkTikRuviGmvrVoS9+f1xonNM-Z==OYA@mail.gmail.com>

Nevermind. In 2.6 (what I'm working with) it's "case IMPORT_NAME:"
Thank you for letting me know where to start.

On 1 June 2011 15:43, Timothy Kadich <tdkadich at gmail.com> wrote:

> I don't understand what you mean by "TARGET(IMPORT_NAME)". I can't find
> that string in ceval.c.
>
>
> On 1 June 2011 12:04, Benjamin Peterson <benjamin at python.org> wrote:
>
>> 2011/6/1 Timothy Kadich <tdkadich at gmail.com>:
>> > If a Python program imported a module, say numpy for example, where in
>> the
>> > source is this line interpreted and then handled by import.c ?
>>
>> Many different files. Start from TARGET(IMPORT_NAME) in ceval.c.
>>
>>
>> --
>> Regards,
>> Benjamin
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110601/1c34b594/attachment-0001.html>

From tjreedy at udel.edu  Thu Jun  2 02:37:52 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 01 Jun 2011 20:37:52 -0400
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Multiple
 clean-ups to the docs for builtin functions.
In-Reply-To: <E1QRuFO-0006aT-Je@dinsdale.python.org>
References: <E1QRuFO-0006aT-Je@dinsdale.python.org>
Message-ID: <4DE6DB60.3030906@udel.edu>

On 6/1/2011 6:50 PM, raymond.hettinger wrote:

> -      with open("mydata.txt") as fp:
> -          for line in iter(fp.readline, "STOP"):
 >                 process_line(line)

As noted on the tracker, this will always loop forever. Even if "STOP"
is corrected to "STOP\n", it will still loop forever if the file does 
not have the magic value.

> +      with open('mydata.txt') as fp:
> +          for line in iter(fp.readline, ''):
 >                 process_line(line)

While I have a big problem with infinite loops, I have a (smaller) 
problem with useless code. The new loop line line is a roundabout way to 
spell "for line in fp". While this would have been useful before files 
were turned into iterables (if there was such a time after iter() was 
introduced), it is not now.

What might be useful and what the example could show is how to stop 
early if a sentinel is present, while still stopping when the iterable 
runs out.  The following alternate fix to the original does just that.

with open("mydata.txt") as fp:
     for line in iter(fp.__next__, "STOP\n"):
          process_line(line)

A tested a runnable variation with and without the exact sentinal. It 
generalizes to *any* iteration than one might want to stop early.

This still has the objection that the loop could be written as
     for line in fp:
         if line == "STOP\n":
             break
          process_line(line)
which is easily generalized to any stopping conditions.

It is hard to think of useful examples of iter(func, sentinal). To be 
sure of someday stopping, func must someday (either certainly or with 
probability 1) either raise StopIteration or produce the sentinal (which 
should therefore be hard to misspell). To be useful, func should not be 
a method of an iterable (or at least not produce the same objects as the 
iterable would when iterated.) It also must produce different values, at 
least sometimes, when called, which means either maintaining internal 
state or pulling values from another source.

Here is a completely different example which meets these criteria. It 
can actually be run (though not doctested ;-). It uses random.randint to 
produce 25 random waiting times for a binary process to hit one of the 
two values.

from random import randint
for i in range(25):
   print(sum(iter(lambda:randint(0,1), 0)), end=',')

The outer loop could be removed, but it hints at how this could be used 
for empirical probability studies.

---
Terry Jan Reedy

From fuzzyman at voidspace.org.uk  Thu Jun  2 11:42:56 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 02 Jun 2011 10:42:56 +0100
Subject: [Python-Dev] Windows download links not working for 2.7.2rc1 and
	3.1.4rc1
Message-ID: <4DE75B20.8000600@voidspace.org.uk>

Hey all,

The links to the Windows downloads for 2.7.2rc1 and 3.1.4rc1 are 404. 
(From the release pages.)

     http://python.org/ftp/python/3.1.3/python-3.1.4rc1.msi
     http://python.org/ftp/python/2.7.1/python-2.7.2rc1.msi

All the best,

Michael Foord

-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From ncoghlan at gmail.com  Thu Jun  2 15:56:07 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 2 Jun 2011 23:56:07 +1000
Subject: [Python-Dev] [Python-checkins] cpython: The compiler class for
	EMX was removed
In-Reply-To: <E1QRpWL-0000J8-Cm@dinsdale.python.org>
References: <E1QRpWL-0000J8-Cm@dinsdale.python.org>
Message-ID: <BANLkTin2Uh7Bu9Ex2LB6=VuW8HsswxPzWw@mail.gmail.com>

On Thu, Jun 2, 2011 at 3:47 AM, eric.araujo <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/c3f8991cd74d
> changeset: ? 70587:c3f8991cd74d
> user: ? ? ? ??ric Araujo <merwok at netwok.org>
> date: ? ? ? ?Wed Jun 01 15:20:44 2011 +0200
> summary:
> ?The compiler class for EMX was removed

This is the kind of checkin comment where the phrasing is a little
confusing. It could mean either "was removed by mistake and has now
been restored" or else "is no longer supported and has been removed".
(Checking the diff resolves the ambiguity in favour of the latter
interpretation, but not all diffs are as simple as this one).

It's worth trying to make the final state of the source tree explicit
in the checkin message (no need to edit the history, just a note for
future reference).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From merwok at netwok.org  Thu Jun  2 16:47:22 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 02 Jun 2011 16:47:22 +0200
Subject: [Python-Dev] [Python-checkins] cpython: The compiler class for
	EMX was removed
In-Reply-To: <BANLkTin2Uh7Bu9Ex2LB6=VuW8HsswxPzWw@mail.gmail.com>
References: <E1QRpWL-0000J8-Cm@dinsdale.python.org>
	<BANLkTin2Uh7Bu9Ex2LB6=VuW8HsswxPzWw@mail.gmail.com>
Message-ID: <4DE7A27A.1010803@netwok.org>

Le 02/06/2011 15:56, Nick Coghlan a ?crit :
> On Thu, Jun 2, 2011 at 3:47 AM, eric.araujo <python-checkins at python.org> wrote:
>>  The compiler class for EMX was removed
> This is the kind of checkin comment where the phrasing is a little
> confusing.

Yep.  Next time I?ll be longer and use something like ?Remove obsolete
mentions of a compiler that was removed?.

From barry at python.org  Thu Jun  2 20:19:00 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 2 Jun 2011 14:19:00 -0400
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <E1QSCKm-0002bl-RU@dinsdale.python.org>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
Message-ID: <20110602141900.7cd024c0@neurotica.wooz.org>

On Jun 02, 2011, at 08:09 PM, guido.van.rossum wrote:

>+    Continuation lines should align wrapped elements either vertically using 
>+    Python's implicit line joining inside parentheses, brackets and braces,
>+    or using a hanging indent of double your code indention, in which case 
>+    there should be no argument on the first line. For example:
>+
>+    Yes:  # Aligned with opening delimiter
>+          foo = long_function_name(var_one, var_two,
>+                                   var_three, var_four)
>+
>+          # Double code indention for hanging indent; nothing on first line
>+          foo = long_function_name(
>+                  var_one, var_two, var_three,
>+                  var_four)
>+
>+    No:   # Stuff on first line forbidden
>+          foo = long_function_name(var_one, var_two,
>+              var_three, var_four)
>+
>+          # 2-space hanging indent forbidden
>+          foo = long_function_name(
>+            var_one, var_two, var_three,
>+            var_four) 

As I mentioned to Guido, I'm not sure where the double-indent recommendation
comes from, but it's entirely possible I missed that discussion.  I agree with
the recommendations, but think a single-indentation level looks fine.  E.g.

        return dict(
            fqdn_listname=member.mailing_list,
            address=member.address.email,
            role=role,
            user=path_to('users/{0}'.format(member.user.user_id)),
            self_link=path_to('members/{0}'.format(member.member_id)),
            )

or

    return b'{0}://{1}:{2}/{3}/{4}'.format(
        ('https' if as_boolean(config.webservice.use_https) else 'http'),
        config.webservice.hostname,
        config.webservice.port,
        config.webservice.api_version,
        (resource[1:] if resource.startswith('/') else resource),
        )

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110602/cd8f1ebe/attachment.pgp>

From v+python at g.nevcal.com  Thu Jun  2 20:50:05 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 02 Jun 2011 11:50:05 -0700
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <20110602141900.7cd024c0@neurotica.wooz.org>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
Message-ID: <4DE7DB5D.9080908@g.nevcal.com>

On 6/2/2011 11:19 AM, Barry Warsaw wrote:
> On Jun 02, 2011, at 08:09 PM, guido.van.rossum wrote:
>
>> +    Continuation lines should align wrapped elements either vertically using
>> +    Python's implicit line joining inside parentheses, brackets and braces,
>> +    or using a hanging indent of double your code indention, in which case
>> +    there should be no argument on the first line. For example:
>> +
>> +    Yes:  # Aligned with opening delimiter
>> +          foo = long_function_name(var_one, var_two,
>> +                                   var_three, var_four)
>> +
>> +          # Double code indention for hanging indent; nothing on first line
>> +          foo = long_function_name(
>> +                  var_one, var_two, var_three,
>> +                  var_four)
>> +
>> +    No:   # Stuff on first line forbidden
>> +          foo = long_function_name(var_one, var_two,
>> +              var_three, var_four)
>> +
>> +          # 2-space hanging indent forbidden
>> +          foo = long_function_name(
>> +            var_one, var_two, var_three,
>> +            var_four)
>
> As I mentioned to Guido, I'm not sure where the double-indent recommendation
> comes from, but it's entirely possible I missed that discussion.  I agree with
> the recommendations, but think a single-indentation level looks fine.  E.g.
>
>          return dict(
>              fqdn_listname=member.mailing_list,
>              address=member.address.email,
>              role=role,
>              user=path_to('users/{0}'.format(member.user.user_id)),
>              self_link=path_to('members/{0}'.format(member.member_id)),
>              )
>
> or
>
>      return b'{0}://{1}:{2}/{3}/{4}'.format(
>          ('https' if as_boolean(config.webservice.use_https) else 'http'),
>          config.webservice.hostname,
>          config.webservice.port,
>          config.webservice.api_version,
>          (resource[1:] if resource.startswith('/') else resource),
>          )
>
> -Barry

One place a double indent is extremely nice is for lines that initiate a 
new indentation, but are themselves continued:

if some_function(
         Some,
         Parameters,
         To,
         Pass,
         )
     If_True_Operations()

is much more readable than:

if some_function(
     Some,
     Parameters,
     To,
     Pass,
     )
     If_True_Operations()



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110602/65859933/attachment.html>

From guido at python.org  Thu Jun  2 21:01:04 2011
From: guido at python.org (Guido van Rossum)
Date: Thu, 2 Jun 2011 12:01:04 -0700
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <4DE7DB5D.9080908@g.nevcal.com>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
	<4DE7DB5D.9080908@g.nevcal.com>
Message-ID: <BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>

Bingo. That's why. (Though you are missing some colons in your examples. :-)

--Guido

On Thu, Jun 2, 2011 at 11:50 AM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On 6/2/2011 11:19 AM, Barry Warsaw wrote:
>
> On Jun 02, 2011, at 08:09 PM, guido.van.rossum wrote:
>
> +    Continuation lines should align wrapped elements either vertically
> using
> +    Python's implicit line joining inside parentheses, brackets and braces,
> +    or using a hanging indent of double your code indention, in which case
> +    there should be no argument on the first line. For example:
> +
> +    Yes:  # Aligned with opening delimiter
> +          foo = long_function_name(var_one, var_two,
> +                                   var_three, var_four)
> +
> +          # Double code indention for hanging indent; nothing on first line
> +          foo = long_function_name(
> +                  var_one, var_two, var_three,
> +                  var_four)
> +
> +    No:   # Stuff on first line forbidden
> +          foo = long_function_name(var_one, var_two,
> +              var_three, var_four)
> +
> +          # 2-space hanging indent forbidden
> +          foo = long_function_name(
> +            var_one, var_two, var_three,
> +            var_four)
>
> As I mentioned to Guido, I'm not sure where the double-indent recommendation
> comes from, but it's entirely possible I missed that discussion.  I agree
> with
> the recommendations, but think a single-indentation level looks fine.  E.g.
>
>         return dict(
>             fqdn_listname=member.mailing_list,
>             address=member.address.email,
>             role=role,
>             user=path_to('users/{0}'.format(member.user.user_id)),
>             self_link=path_to('members/{0}'.format(member.member_id)),
>             )
>
> or
>
>     return b'{0}://{1}:{2}/{3}/{4}'.format(
>         ('https' if as_boolean(config.webservice.use_https) else 'http'),
>         config.webservice.hostname,
>         config.webservice.port,
>         config.webservice.api_version,
>         (resource[1:] if resource.startswith('/') else resource),
>         )
>
> -Barry
>
> One place a double indent is extremely nice is for lines that initiate a new
> indentation, but are themselves continued:
>
> if some_function(
> ??????? Some,
> ??????? Parameters,
> ??????? To,
> ??????? Pass,
> ??????? )
> ??? If_True_Operations()
>
> is much more readable than:
>
> if some_function(
> ??? Some,
> ??? Parameters,
> ??? To,
> ??? Pass,
> ??? )
> ??? If_True_Operations()
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (python.org/~guido)

From rdmurray at bitdance.com  Thu Jun  2 21:07:06 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 02 Jun 2011 15:07:06 -0400
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
	continuation lines.
In-Reply-To: <20110602141900.7cd024c0@neurotica.wooz.org>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
Message-ID: <20110602190707.39C6425059F@webabinitio.net>

On Thu, 02 Jun 2011 14:19:00 -0400, Barry Warsaw <barry at python.org> wrote:
> On Jun 02, 2011, at 08:09 PM, guido.van.rossum wrote:
> >+    Yes:  # Aligned with opening delimiter
> >+          foo = long_function_name(var_one, var_two,
> >+                                   var_three, var_four)
> >+
> >+          # Double code indention for hanging indent; nothing on first li=
> ne
> >+          foo = long_function_name(
> >+                  var_one, var_two, var_three,
> >+                  var_four)
> >+
> >+    No:   # Stuff on first line forbidden
> >+          foo = long_function_name(var_one, var_two,
> >+              var_three, var_four)
> >+
> >+          # 2-space hanging indent forbidden
> >+          foo = long_function_name(
> >+            var_one, var_two, var_three,
> >+            var_four)
> 
> As I mentioned to Guido, I'm not sure where the double-indent recommendation
> comes from, but it's entirely possible I missed that discussion.  I agree with
> the recommendations, but think a single-indentation level looks fine.  E.g.

Personally, I use "enough" indentation.  Sometimes that is a single
indentation level, but sometimes it is more.  Two spaces is definitely
right out, though :)

The place where a single indentation level is *not* enough is when the
line being indented is the statement starting a suite:

    for x in long_function_name(
        var_one, var_two, var_three):
        print x

    vs

    for x in long_function_name(
            var_one, var_two, var_three):
        print x

That's a case where I'd be likely to use even more than two indentation
levels.  Usually, though, I try to refactor the statement so it fits
on one line.

--
R. David Murray           http://www.bitdance.com

From martin at v.loewis.de  Thu Jun  2 21:39:21 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 02 Jun 2011 21:39:21 +0200
Subject: [Python-Dev] Windows download links not working for 2.7.2rc1
 and	3.1.4rc1
In-Reply-To: <4DE75B20.8000600@voidspace.org.uk>
References: <4DE75B20.8000600@voidspace.org.uk>
Message-ID: <4DE7E6E9.2020300@v.loewis.de>

> The links to the Windows downloads for 2.7.2rc1 and 3.1.4rc1 are 404.
> (From the release pages.)

Thanks, fixed.

Martin

From barry at python.org  Thu Jun  2 21:42:30 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 2 Jun 2011 15:42:30 -0400
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <20110602190707.39C6425059F@webabinitio.net>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
	<20110602190707.39C6425059F@webabinitio.net>
Message-ID: <20110602154230.04087942@limelight.wooz.org>

On Jun 02, 2011, at 03:07 PM, R. David Murray wrote:

>Personally, I use "enough" indentation.  Sometimes that is a single
>indentation level, but sometimes it is more.  Two spaces is definitely
>right out, though :)
>
>The place where a single indentation level is *not* enough is when the
>line being indented is the statement starting a suite:
>
>    for x in long_function_name(
>        var_one, var_two, var_three):
>        print x
>
>    vs
>
>    for x in long_function_name(
>            var_one, var_two, var_three):
>        print x
>
>That's a case where I'd be likely to use even more than two indentation
>levels.  Usually, though, I try to refactor the statement so it fits
>on one line.

That's a recommendation I can definitely get on board with!  I do exactly the
same thing, although sometimes I use a comment between the compound statement
and the indented stanza.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110602/b8f4f558/attachment.pgp>

From v+python at g.nevcal.com  Thu Jun  2 21:57:07 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 02 Jun 2011 12:57:07 -0700
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
	<4DE7DB5D.9080908@g.nevcal.com>
	<BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
Message-ID: <4DE7EB13.30600@g.nevcal.com>

On 6/2/2011 12:01 PM, Guido van Rossum wrote:
> Bingo. That's why. (Though you are missing some colons in your examples.:-)
>
> --Guido

You operate as a good Python compiler :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110602/bda0e17c/attachment.html>

From barry at python.org  Fri Jun  3 00:15:58 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 2 Jun 2011 18:15:58 -0400
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <4DE7EB13.30600@g.nevcal.com>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
	<4DE7DB5D.9080908@g.nevcal.com>
	<BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
	<4DE7EB13.30600@g.nevcal.com>
Message-ID: <20110602181558.53cc8616@neurotica.wooz.org>

On Jun 02, 2011, at 12:57 PM, Glenn Linderman wrote:

>On 6/2/2011 12:01 PM, Guido van Rossum wrote:
>> Bingo. That's why. (Though you are missing some colons in your examples.:-)
>>
>> --Guido
>
>You operate as a good Python compiler :)

Actually, this is a key insight, which I just mentioned in a private response
to Steve.  How about this for the rule:

If the hanging line ends in a colon, then double-indent (or "more-ly
indented") seems appropriate.  If not, then a single indent is sufficient.

That handles cases like this, where double indent makes sense:

def some_really_long_function_name(
        an_argument,
        another_argument,
        and_a_third_argument):
    foo()

---

    if some_really_long_function_name(
            an_argument,
            another_argument,
            and_a_third_argument):
        foo()

---

and these cases where a single indent is fine:

    x = some_really_long_function_name(
        an_argument,
        another_argument,
        and_a_third_argument)
    foo(x)

---

    d = dict(
        a=1,
        b=2,
        c=3,
        d=4)
    return d

Does that cover it?
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110602/63efc101/attachment.pgp>

From guido at python.org  Fri Jun  3 00:49:08 2011
From: guido at python.org (Guido van Rossum)
Date: Thu, 2 Jun 2011 15:49:08 -0700
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <20110602181558.53cc8616@neurotica.wooz.org>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
	<4DE7DB5D.9080908@g.nevcal.com>
	<BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
	<4DE7EB13.30600@g.nevcal.com>
	<20110602181558.53cc8616@neurotica.wooz.org>
Message-ID: <BANLkTi=hxHHX=GfnBkXZ8thjrCBNPJ+H2Q@mail.gmail.com>

On Thu, Jun 2, 2011 at 3:15 PM, Barry Warsaw <barry at python.org> wrote:
> On Jun 02, 2011, at 12:57 PM, Glenn Linderman wrote:
>
>>On 6/2/2011 12:01 PM, Guido van Rossum wrote:
>>> Bingo. That's why. (Though you are missing some colons in your examples.:-)
>>>
>>> --Guido
>>
>>You operate as a good Python compiler :)
>
> Actually, this is a key insight, which I just mentioned in a private response
> to Steve. ?How about this for the rule:
>
> If the hanging line ends in a colon, then double-indent (or "more-ly
> indented") seems appropriate. ?If not, then a single indent is sufficient.
>
> That handles cases like this, where double indent makes sense:
>
> def some_really_long_function_name(
> ? ? ? ?an_argument,
> ? ? ? ?another_argument,
> ? ? ? ?and_a_third_argument):
> ? ?foo()
>
> ---
>
> ? ?if some_really_long_function_name(
> ? ? ? ? ? ?an_argument,
> ? ? ? ? ? ?another_argument,
> ? ? ? ? ? ?and_a_third_argument):
> ? ? ? ?foo()
>
> ---
>
> and these cases where a single indent is fine:
>
> ? ?x = some_really_long_function_name(
> ? ? ? ?an_argument,
> ? ? ? ?another_argument,
> ? ? ? ?and_a_third_argument)
> ? ?foo(x)
>
> ---
>
> ? ?d = dict(
> ? ? ? ?a=1,
> ? ? ? ?b=2,
> ? ? ? ?c=3,
> ? ? ? ?d=4)
> ? ?return d
>
> Does that cover it?

Except that the rule gets more complicated. I don't think that always
using the double indent is going to mean a lot more line breaks, so I
don't think there's much benefit to the added complication.

-- 
--Guido van Rossum (python.org/~guido)

From v+python at g.nevcal.com  Fri Jun  3 01:16:30 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 02 Jun 2011 16:16:30 -0700
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <BANLkTi=hxHHX=GfnBkXZ8thjrCBNPJ+H2Q@mail.gmail.com>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
	<4DE7DB5D.9080908@g.nevcal.com>
	<BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
	<4DE7EB13.30600@g.nevcal.com>
	<20110602181558.53cc8616@neurotica.wooz.org>
	<BANLkTi=hxHHX=GfnBkXZ8thjrCBNPJ+H2Q@mail.gmail.com>
Message-ID: <4DE819CE.700@g.nevcal.com>

On 6/2/2011 3:49 PM, Guido van Rossum wrote:
> Except that the rule gets more complicated. I don't think that always
> using the double indent is going to mean a lot more line breaks, so I
> don't think there's much benefit to the added complication.

Further, tools like python-mode would have to go back and fix the indent 
to be double indented when there are multiple lines, and the : is not 
typed until the last line... not impossible, but more complicated, and 
some of the intervening lines might be too long for the double indent, 
and then manual adjustments would have to happen.  Ick.

Double indent from the beginning, if there is nothing after the (.

Or key the double indent off the leading keywords that end with :

Here's a language question, though: if there are keywords that imply the 
need for a :, and it isn't on the same line, why is \ needed to continue 
to the next line (or parentheses, etc.)?

If the : is truly omitted, like I did in my example, there'll be other 
syntax errors to report.  If it is just on a later line, why complain 
about it missing?

With complex conditions, I wind up adding extra () rather than trailing 
\, and that is about the only time I have ever found the need to use \ 
(but my workaround is to add the ()).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110602/bf4f0d03/attachment-0001.html>

From greg.ewing at canterbury.ac.nz  Fri Jun  3 00:18:53 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Jun 2011 10:18:53 +1200
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
	<4DE7DB5D.9080908@g.nevcal.com>
	<BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
Message-ID: <4DE80C4D.4070808@canterbury.ac.nz>

Guido van Rossum wrote:
> Bingo. That's why. (Though you are missing some colons in your examples. :-)
> 
> --Guido
> 
> On Thu, Jun 2, 2011 at 11:50 AM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> 
>>One place a double indent is extremely nice is for lines that initiate a new
>>indentation, but are themselves continued:
>>
>>if some_function(
>>        Some,
>>        Parameters,
>>        To,
>>        Pass,
>>        )
>>    If_True_Operations()

Another way to approach that is

if some_function(
     Some,
     Parameters,
     To,
     Pass,
     ):
         If_True_Operations()

i.e. indent the *body* one more place. This avoids the
jarriness of seeing an outdent that doesn't correspond
to the closing of a suite.

-- 
Greg

From v+python at g.nevcal.com  Fri Jun  3 03:35:42 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 02 Jun 2011 18:35:42 -0700
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <4DE80C4D.4070808@canterbury.ac.nz>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>	<20110602141900.7cd024c0@neurotica.wooz.org>	<4DE7DB5D.9080908@g.nevcal.com>	<BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
	<4DE80C4D.4070808@canterbury.ac.nz>
Message-ID: <4DE83A6E.6020709@g.nevcal.com>

On 6/2/2011 3:18 PM, Greg Ewing wrote:
> i.e. indent the *body* one more place. This avoids the
> jarriness of seeing an outdent that doesn't correspond
> to the closing of a suite. 

-1.  There are likely many more lines in the suite than in the 
conditional, that, by being double indented, would now need 
continuations themselves.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110602/1b507f33/attachment.html>

From ben+python at benfinney.id.au  Fri Jun  3 04:16:37 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 03 Jun 2011 12:16:37 +1000
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
	continuation lines.
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
	<4DE7DB5D.9080908@g.nevcal.com>
	<BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
	<4DE80C4D.4070808@canterbury.ac.nz>
Message-ID: <8739jr64my.fsf@benfinney.id.au>

Greg Ewing <greg.ewing at canterbury.ac.nz> writes:

> Another way to approach that is
>
> if some_function(
>     Some,
>     Parameters,
>     To,
>     Pass,
>     ):
>         If_True_Operations()
>
> i.e. indent the *body* one more place. This avoids the jarriness of
> seeing an outdent that doesn't correspond to the closing of a suite.

?1. Continuation lines for a single statement *should* look different (I
disagree with ?jarring?) from a suite of statements.

-- 
 \         ?Simplicity and elegance are unpopular because they require |
  `\           hard work and discipline to achieve and education to be |
_o__)                                appreciated.? ?Edsger W. Dijkstra |
Ben Finney


From regebro at gmail.com  Fri Jun  3 12:09:34 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Fri, 3 Jun 2011 12:09:34 +0200
Subject: [Python-Dev] [Python-checkins] peps: Add rules for indenting
 continuation lines.
In-Reply-To: <20110602181558.53cc8616@neurotica.wooz.org>
References: <E1QSCKm-0002bl-RU@dinsdale.python.org>
	<20110602141900.7cd024c0@neurotica.wooz.org>
	<4DE7DB5D.9080908@g.nevcal.com>
	<BANLkTi=ed19WMXzz67XfzpXZJoJEM9_b1g@mail.gmail.com>
	<4DE7EB13.30600@g.nevcal.com>
	<20110602181558.53cc8616@neurotica.wooz.org>
Message-ID: <BANLkTinn+qmv_LrT-dVZ-bYTiMdpPEf8SA@mail.gmail.com>

On Fri, Jun 3, 2011 at 00:15, Barry Warsaw <barry at python.org> wrote:
> On Jun 02, 2011, at 12:57 PM, Glenn Linderman wrote:
>
>>On 6/2/2011 12:01 PM, Guido van Rossum wrote:
>>> Bingo. That's why. (Though you are missing some colons in your examples.:-)
>>>
>>> --Guido
>>
>>You operate as a good Python compiler :)
>
> Actually, this is a key insight, which I just mentioned in a private response
> to Steve. ?How about this for the rule:
>
> If the hanging line ends in a colon, then double-indent (or "more-ly
> indented") seems appropriate. ?If not, then a single indent is sufficient.
>
> That handles cases like this, where double indent makes sense:
>
> def some_really_long_function_name(
> ? ? ? ?an_argument,
> ? ? ? ?another_argument,
> ? ? ? ?and_a_third_argument):
> ? ?foo()
>
> ---
>
> ? ?if some_really_long_function_name(
> ? ? ? ? ? ?an_argument,
> ? ? ? ? ? ?another_argument,
> ? ? ? ? ? ?and_a_third_argument):
> ? ? ? ?foo()
>
> ---
>
> and these cases where a single indent is fine:
>
> ? ?x = some_really_long_function_name(
> ? ? ? ?an_argument,
> ? ? ? ?another_argument,
> ? ? ? ?and_a_third_argument)
> ? ?foo(x)
>
> ---
>
> ? ?d = dict(
> ? ? ? ?a=1,
> ? ? ? ?b=2,
> ? ? ? ?c=3,
> ? ? ? ?d=4)
> ? ?return d
>
> Does that cover it?

I like it.

From status at bugs.python.org  Fri Jun  3 18:07:23 2011
From: status at bugs.python.org (Python tracker)
Date: Fri,  3 Jun 2011 18:07:23 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20110603160723.721B91CB61@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2011-05-27 - 2011-06-03)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2815 ( +2)
  closed 21221 (+56)
  total  24036 (+58)

Open issues with patches: 1224 


Issues opened (38)
==================

#12197: non-blocking SSL write in Windows sends large data but raises 
http://bugs.python.org/issue12197  opened by dsiroky

#12198: zipfile.py:1047: DeprecationWarning: 'H' format requires 0 <= 
http://bugs.python.org/issue12198  opened by gnezdo

#12200: bdist_wininst install_script not run on uninstall
http://bugs.python.org/issue12200  opened by mhammond

#12201: Returning FILETIME is unsupported in msilib.SummaryInformation
http://bugs.python.org/issue12201  opened by markm

#12202: Check status returns in msilib.SummaryInformation.GetProperty(
http://bugs.python.org/issue12202  opened by markm

#12204: str.upper converts to title
http://bugs.python.org/issue12204  opened by py.user

#12206: Documentation Std. Library 15.7.5 "LogRecord objects": Paramet
http://bugs.python.org/issue12206  opened by mponweiser

#12207: Document ast.PyCF_ONLY_AST
http://bugs.python.org/issue12207  opened by eric.araujo

#12208: Glitches in email.policy docs
http://bugs.python.org/issue12208  opened by eric.araujo

#12209: Minor edits to faulthandler doc
http://bugs.python.org/issue12209  opened by eric.araujo

#12210: test_smtplib: intermittent failures on FreeBSD
http://bugs.python.org/issue12210  opened by skrah

#12211: math.copysign must keep object type.
http://bugs.python.org/issue12211  opened by umedoblock

#12213: BufferedRandom, BufferedRWPair: issues with interlaced read-wr
http://bugs.python.org/issue12213  opened by haypo

#12214: platform module can't detect archlinux distribution
http://bugs.python.org/issue12214  opened by lilaboc

#12215: TextIOWrapper: issues with interlaced read-write
http://bugs.python.org/issue12215  opened by haypo

#12217: Cross-link docs for faulthandler, traceback and pdb
http://bugs.python.org/issue12217  opened by eric.araujo

#12219: tkinter.filedialog.askopenfilename XT dialog on Windows 7
http://bugs.python.org/issue12219  opened by hfischer

#12220: minidom xmlns not handling spaces in xmlns attribute value fie
http://bugs.python.org/issue12220  opened by hfischer

#12222: All pysetup commands should respect exit codes
http://bugs.python.org/issue12222  opened by tarek

#12223: Datamodel documentation page: 'operator' where 'operand' shoul
http://bugs.python.org/issue12223  opened by Vladimir.M.

#12224: problem with siginterrupt
http://bugs.python.org/issue12224  opened by Zhiping.Deng

#12226: use secured channel for uploading packages to pypi
http://bugs.python.org/issue12226  opened by techtonik

#12228: Stat doc - fix swap of UF_OPAQUE and UF_NOUNLINK description
http://bugs.python.org/issue12228  opened by sandro.tosi

#12231: regrtest: add -k and -K options to filter tests by function/fi
http://bugs.python.org/issue12231  opened by haypo

#12232: embedded python import cmath
http://bugs.python.org/issue12232  opened by Thanh Ly

#12234: unittest2 could enable regex debugging for more information
http://bugs.python.org/issue12234  opened by iElectric

#12237: Document how to open non-default webbrowser
http://bugs.python.org/issue12237  opened by hirsh

#12238: Readline module loading in interactive mode
http://bugs.python.org/issue12238  opened by Niels.Heinen

#12239: msilib VT_EMPTY SummaryInformation properties raise an error (
http://bugs.python.org/issue12239  opened by markm

#12240: Allow multiple setup_hooks
http://bugs.python.org/issue12240  opened by erik.bray

#12242: distutils2 environment marker for current compiler
http://bugs.python.org/issue12242  opened by eli.collins

#12243: getpass.getuser works on OSX
http://bugs.python.org/issue12243  opened by louiscipher

#12245: Document the meaning of FLT_ROUNDS constants for sys.float_inf
http://bugs.python.org/issue12245  opened by rhettinger

#12246: create installation path if it's non-existent
http://bugs.python.org/issue12246  opened by tshepang

#12248: __dir__ semantics changed in Python 2.7.2
http://bugs.python.org/issue12248  opened by barry

#12249: add missing command
http://bugs.python.org/issue12249  opened by tshepang

#12253: Document packaging.manifest and packaging.errors
http://bugs.python.org/issue12253  opened by eric.araujo

#12254: PEP-3107 has a wrong attribute name for function annotations
http://bugs.python.org/issue12254  opened by Roman.Alexeev



Most recent 15 issues with no replies (15)
==========================================

#12254: PEP-3107 has a wrong attribute name for function annotations
http://bugs.python.org/issue12254

#12253: Document packaging.manifest and packaging.errors
http://bugs.python.org/issue12253

#12245: Document the meaning of FLT_ROUNDS constants for sys.float_inf
http://bugs.python.org/issue12245

#12242: distutils2 environment marker for current compiler
http://bugs.python.org/issue12242

#12239: msilib VT_EMPTY SummaryInformation properties raise an error (
http://bugs.python.org/issue12239

#12232: embedded python import cmath
http://bugs.python.org/issue12232

#12228: Stat doc - fix swap of UF_OPAQUE and UF_NOUNLINK description
http://bugs.python.org/issue12228

#12223: Datamodel documentation page: 'operator' where 'operand' shoul
http://bugs.python.org/issue12223

#12220: minidom xmlns not handling spaces in xmlns attribute value fie
http://bugs.python.org/issue12220

#12217: Cross-link docs for faulthandler, traceback and pdb
http://bugs.python.org/issue12217

#12214: platform module can't detect archlinux distribution
http://bugs.python.org/issue12214

#12208: Glitches in email.policy docs
http://bugs.python.org/issue12208

#12201: Returning FILETIME is unsupported in msilib.SummaryInformation
http://bugs.python.org/issue12201

#12197: non-blocking SSL write in Windows sends large data but raises 
http://bugs.python.org/issue12197

#12179: Race condition using PyGILState_Ensure on a new thread
http://bugs.python.org/issue12179



Most recent 15 issues waiting for review (15)
=============================================

#12249: add missing command
http://bugs.python.org/issue12249

#12246: create installation path if it's non-existent
http://bugs.python.org/issue12246

#12243: getpass.getuser works on OSX
http://bugs.python.org/issue12243

#12231: regrtest: add -k and -K options to filter tests by function/fi
http://bugs.python.org/issue12231

#12228: Stat doc - fix swap of UF_OPAQUE and UF_NOUNLINK description
http://bugs.python.org/issue12228

#12226: use secured channel for uploading packages to pypi
http://bugs.python.org/issue12226

#12215: TextIOWrapper: issues with interlaced read-write
http://bugs.python.org/issue12215

#12214: platform module can't detect archlinux distribution
http://bugs.python.org/issue12214

#12213: BufferedRandom, BufferedRWPair: issues with interlaced read-wr
http://bugs.python.org/issue12213

#12210: test_smtplib: intermittent failures on FreeBSD
http://bugs.python.org/issue12210

#12209: Minor edits to faulthandler doc
http://bugs.python.org/issue12209

#12208: Glitches in email.policy docs
http://bugs.python.org/issue12208

#12207: Document ast.PyCF_ONLY_AST
http://bugs.python.org/issue12207

#12202: Check status returns in msilib.SummaryInformation.GetProperty(
http://bugs.python.org/issue12202

#12201: Returning FILETIME is unsupported in msilib.SummaryInformation
http://bugs.python.org/issue12201



Top 10 most discussed issues (10)
=================================

#11975: Fix referencing of built-in types (list, int, ...)
http://bugs.python.org/issue11975  19 msgs

#11416: netrc module does not handle multiple entries for a single hos
http://bugs.python.org/issue11416  13 msgs

#12226: use secured channel for uploading packages to pypi
http://bugs.python.org/issue12226  12 msgs

#12185: Decimal documentation lists "first" and "second" arguments, sh
http://bugs.python.org/issue12185  10 msgs

#11623: Distutils is reporting OSX 10.6 w/ XCode 4 as "universal"
http://bugs.python.org/issue11623   9 msgs

#11906: test_argparse failure in interactive mode
http://bugs.python.org/issue11906   8 msgs

#12082: Python/import.c still references fstat even with DONT_HAVE_FST
http://bugs.python.org/issue12082   7 msgs

#6490: os.popen documentation in 2.6 is probably wrong
http://bugs.python.org/issue6490   5 msgs

#12191: Add shutil.chown to allow to use user and group name (and not 
http://bugs.python.org/issue12191   5 msgs

#12246: create installation path if it's non-existent
http://bugs.python.org/issue12246   5 msgs



Issues closed (47)
==================

#3265: Python-2.5.2/Modules/_ctypes/malloc_closure.c:70: error: `MAP_
http://bugs.python.org/issue3265  closed by terry.reedy

#8578: PyWeakref_GetObject
http://bugs.python.org/issue8578  closed by python-dev

#8678: crashers in rgbimg
http://bugs.python.org/issue8678  closed by terry.reedy

#9382: os.popen referenced but not documented in Python 3.x
http://bugs.python.org/issue9382  closed by eric.araujo

#9738: Document the encoding of functions bytes arguments of the C AP
http://bugs.python.org/issue9738  closed by haypo

#9812: cPickle segfault with nested dicts in threaded env
http://bugs.python.org/issue9812  closed by terry.reedy

#10449: ???os.environ was modified by test_httpservers???
http://bugs.python.org/issue10449  closed by eric.araujo

#10616: Change PyObject_AsCharBuffer() error message
http://bugs.python.org/issue10616  closed by python-dev

#11217: python-32 not linked in /usr/local/bin in framework builds
http://bugs.python.org/issue11217  closed by ned.deily

#11449: tarfile tries to file_.tell() even when creating a new archive
http://bugs.python.org/issue11449  closed by strombrg

#11612: xml.dom.minidom fail to parse SVG file.
http://bugs.python.org/issue11612  closed by ned.deily

#11864: sporadic failure in test_concurrent_futures
http://bugs.python.org/issue11864  closed by bquinlan

#12028: threading._get_ident(): remove it in the doc or make it public
http://bugs.python.org/issue12028  closed by python-dev

#12057: HZ codec has no test
http://bugs.python.org/issue12057  closed by haypo

#12085: subprocess.Popen.__del__ raises AttributeError if __init__ was
http://bugs.python.org/issue12085  closed by python-dev

#12089: regrtest.py doesn't check for unexpected output anymore?
http://bugs.python.org/issue12089  closed by python-dev

#12101: PEPs should have consecutive revision numbers
http://bugs.python.org/issue12101  closed by eric.araujo

#12106: reflect syntatic sugar in with ast
http://bugs.python.org/issue12106  closed by python-dev

#12125: test_sysconfig fails on OpenIndiana because of test_packaging
http://bugs.python.org/issue12125  closed by tarek

#12151: test_logging fails sometimes
http://bugs.python.org/issue12151  closed by vinay.sajip

#12171: Reset method of the incremental encoders of CJK codecs	calls t
http://bugs.python.org/issue12171  closed by python-dev

#12175: FileIO.readall() read the file position and size at each read
http://bugs.python.org/issue12175  closed by haypo

#12190: intern filenames in bytecode
http://bugs.python.org/issue12190  closed by benjamin.peterson

#12196: add pipe2() to the os module
http://bugs.python.org/issue12196  closed by charles-francois.natali

#12199: Unify TryExcept and TryFinally
http://bugs.python.org/issue12199  closed by python-dev

#12203: isinstance not functioning as documented
http://bugs.python.org/issue12203  closed by benjamin.peterson

#12205: test_subprocess fails due to uninstalled test subdirectory
http://bugs.python.org/issue12205  closed by ned.deily

#12212: Minor proofreading typo in index.rst
http://bugs.python.org/issue12212  closed by benjamin.peterson

#12216: future imports change the reporting of syntaxerrors
http://bugs.python.org/issue12216  closed by python-dev

#12218: Removing wsgiref.egg-info
http://bugs.python.org/issue12218  closed by tarek

#12221: segfaults with unexpanded $Revision$ id's in release candidate
http://bugs.python.org/issue12221  closed by python-dev

#12225: current tip doesn't build without mercurial installed
http://bugs.python.org/issue12225  closed by r.david.murray

#12227: multiprocessing.Pool initialization seems to call methods it s
http://bugs.python.org/issue12227  closed by Graham.Cummins

#12229: Remove unused variable in bufferedio.c
http://bugs.python.org/issue12229  closed by python-dev

#12230: test_subprocess.test_pass_fds() failed on x86 Tiger 3.x
http://bugs.python.org/issue12230  closed by python-dev

#12233: Lists in class objects not independent
http://bugs.python.org/issue12233  closed by rhettinger

#12235: subprocess throws wrong exception if script can't be executed
http://bugs.python.org/issue12235  closed by r.david.murray

#12236: Tkinter __version__ uses subversion substitution
http://bugs.python.org/issue12236  closed by benjamin.peterson

#12241: email.header.Header encoding fails on empty header
http://bugs.python.org/issue12241  closed by r.david.murray

#12244: cStringIO inconsistencies
http://bugs.python.org/issue12244  closed by georg.brandl

#12247: Finding subprocess.py depends on how IDLE is opened
http://bugs.python.org/issue12247  closed by ned.deily

#12250: regrtest: make --timeout explicit
http://bugs.python.org/issue12250  closed by python-dev

#12251: subprocess(..., stdout=sys.stderr) closes stderr for child
http://bugs.python.org/issue12251  closed by charles-francois.natali

#12252: '\u' in unicode raw string raise an syntax error
http://bugs.python.org/issue12252  closed by amaury.forgeotdarc

#1495802: cygwin: popen3 lock up
http://bugs.python.org/issue1495802  closed by charles-francois.natali

#1189811: pydoc may hide non-private doc strings.
http://bugs.python.org/issue1189811  closed by ncoghlan

#985064: plistlib crashes too easily on bad files
http://bugs.python.org/issue985064  closed by ned.deily

From rosslagerwall at gmail.com  Fri Jun  3 18:34:01 2011
From: rosslagerwall at gmail.com (Ross Lagerwall)
Date: Fri, 03 Jun 2011 18:34:01 +0200
Subject: [Python-Dev] os.stat and nanosecond precision
In-Reply-To: <20110526060903.GB7580@colossus>
References: <BANLkTikXottB8xfXbe1y48P-GWVxhKn2=Q@mail.gmail.com>
	<BANLkTi=8K1=ZQfxAD0nK0wAaCy33FRmEEw@mail.gmail.com>
	<1306343861.20117.4.camel@marge>
	<BANLkTikHWFa+0K7kbRbk0suXDM=cELke-A@mail.gmail.com>
	<BANLkTimTMVTgW6f1LnSgFq3a1ugqZywJhA@mail.gmail.com>
	<20110526060903.GB7580@colossus>
Message-ID: <1307118841.1447.6.camel@hobo>

With regards to http://bugs.python.org/issue11457

What should the name of the (seconds, nanoseconds) tuple be?
st_atim, st_ctim and st_mtim has bee suggested and is what the POSIX
specification uses. This is confusingly similar to the existing
st_atime, st_ctime and st_mtime.

Also, should it be that these attributes are always defined (and just
have 0 for nanoseconds if the OS doesn't support it) or should
it be that they are only defined if the OS supports nanosecond
precision.
If they are always defined, it would make usage simpler.

Cheers
Ross


From alexander.belopolsky at gmail.com  Fri Jun  3 18:52:03 2011
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 3 Jun 2011 12:52:03 -0400
Subject: [Python-Dev] os.stat and nanosecond precision
In-Reply-To: <1307118841.1447.6.camel@hobo>
References: <BANLkTikXottB8xfXbe1y48P-GWVxhKn2=Q@mail.gmail.com>
	<BANLkTi=8K1=ZQfxAD0nK0wAaCy33FRmEEw@mail.gmail.com>
	<1306343861.20117.4.camel@marge>
	<BANLkTikHWFa+0K7kbRbk0suXDM=cELke-A@mail.gmail.com>
	<BANLkTimTMVTgW6f1LnSgFq3a1ugqZywJhA@mail.gmail.com>
	<20110526060903.GB7580@colossus> <1307118841.1447.6.camel@hobo>
Message-ID: <BANLkTi=sUy8d8uCkVN2+0NzNSTLFJ11QKQ@mail.gmail.com>

On Fri, Jun 3, 2011 at 12:34 PM, Ross Lagerwall <rosslagerwall at gmail.com> wrote:
> ..
> What should the name of the (seconds, nanoseconds) tuple be?
> st_atim, st_ctim and st_mtim has bee suggested and is what the POSIX
> specification uses. This is confusingly similar to the existing
> st_atime, st_ctime and st_mtime.
>
Still, inventing new names would make it even more confusing.

+1 for POSIX names.


> Also, should it be that these attributes are always defined (and just
> have 0 for nanoseconds if the OS doesn't support it) or should
> it be that they are only defined if the OS supports nanosecond
> precision.
> If they are always defined, it would make usage simpler.

+1 to have them always defined.

From raymond.hettinger at gmail.com  Fri Jun  3 19:43:14 2011
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Fri, 3 Jun 2011 10:43:14 -0700
Subject: [Python-Dev] [Python-checkins] cpython: Fix reST label for
	collections ABCs.
In-Reply-To: <E1QSY9n-00048c-4A@dinsdale.python.org>
References: <E1QSY9n-00048c-4A@dinsdale.python.org>
Message-ID: <1AA56E88-906F-455D-AE11-510E07CCE04A@gmail.com>


On Jun 3, 2011, at 10:27 AM, eric.araujo wrote:
> 
>  Fix reST label for collections ABCs.
> 
> The previous markup hijacked the abstract-base-classes glossary entry,
> which resulted in the HTML linking to collections.abc when defining the
> generic ABC concept.  Now the glossary links to the abc module.

I think the users are better served by links to collections.abc, io.abc, etc.
For the most part, glossary readers will be interested in actual
abstract classes than in the underlying implementation.

IOW, I believe this edit makes the docs worse rather than better.


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110603/43b3b816/attachment.html>

From merwok at netwok.org  Fri Jun  3 19:47:32 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 03 Jun 2011 19:47:32 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Fix reST label for
 collections ABCs.
In-Reply-To: <1AA56E88-906F-455D-AE11-510E07CCE04A@gmail.com>
References: <E1QSY9n-00048c-4A@dinsdale.python.org>
	<1AA56E88-906F-455D-AE11-510E07CCE04A@gmail.com>
Message-ID: <4DE91E34.2070602@netwok.org>

Le 03/06/2011 19:43, Raymond Hettinger a ?crit :
> I think the users are better served by links to collections.abc, io.abc, etc.

The specific problem I addressed was that :ref:`abstract-base-classes`
was replaced by ?Collections Abstract Base Classes?, which was wrong:
the glossary entry talks about ?Abstract Base Classes?.

I will add links to abc submodules as you suggested tomorrow.

Thanks for the review.

From martin at v.loewis.de  Fri Jun  3 22:08:47 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 03 Jun 2011 22:08:47 +0200
Subject: [Python-Dev] os.stat and nanosecond precision
In-Reply-To: <1307118841.1447.6.camel@hobo>
References: <BANLkTikXottB8xfXbe1y48P-GWVxhKn2=Q@mail.gmail.com>	<BANLkTi=8K1=ZQfxAD0nK0wAaCy33FRmEEw@mail.gmail.com>	<1306343861.20117.4.camel@marge>	<BANLkTikHWFa+0K7kbRbk0suXDM=cELke-A@mail.gmail.com>	<BANLkTimTMVTgW6f1LnSgFq3a1ugqZywJhA@mail.gmail.com>	<20110526060903.GB7580@colossus>
	<1307118841.1447.6.camel@hobo>
Message-ID: <4DE93F4F.7000004@v.loewis.de>

> What should the name of the (seconds, nanoseconds) tuple be?

See my comment: -1 on having such a tuple in the first place.
We have the decimal type to represent arbitrary-precision time stamps.

> st_atim, st_ctim and st_mtim has bee suggested and is what the POSIX
> specification uses. This is confusingly similar to the existing
> st_atime, st_ctime and st_mtime.

That the POSIX spec uses it is IMO a strong argument.

> Also, should it be that these attributes are always defined (and just
> have 0 for nanoseconds if the OS doesn't support it)

Definitely. There may be a day when ps, fs, or as resolution comes
along. It would be good if a) the fields are always available, and
b) have "unspecified"/"best-effort" precision, to accomodate future
changes.

I wish the decimal type would have been available in 2.3, when
I changed the fields to be doubles instead of longs...

Regards,
Martin

From martin at v.loewis.de  Fri Jun  3 23:40:21 2011
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 03 Jun 2011 23:40:21 +0200
Subject: [Python-Dev] Sniffing passwords from PyPI using insecure
	connection
In-Reply-To: <20110601070818.244edc03@neurotica.wooz.org>
References: <BANLkTikfTXNrvBQpJ-_kXHePon7ynAmOGw@mail.gmail.com>	<is3e5o$qou$1@dough.gmane.org>
	<4DE5D005.5000001@v.loewis.de>	<is4mgf$6uh$1@dough.gmane.org>
	<20110601070818.244edc03@neurotica.wooz.org>
Message-ID: <4DE954C5.7080609@v.loewis.de>

> I followed up on the tracker.  I'm +0 on adding this to 2.6, but not until
> after the 2.6.7 release on Friday.
> 
> How well has this change been tested?  Are there people for whom this could
> break things?

As others have pointed out: it would break systems that don't have the
_ssl module built.

Regards,
Martin

From victor.stinner at haypocalc.com  Sat Jun  4 00:06:11 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 4 Jun 2011 00:06:11 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Add NEWS and whatsnew
	entries for the packaging module
In-Reply-To: <E1QSWIq-0002aZ-DD@dinsdale.python.org>
References: <E1QSWIq-0002aZ-DD@dinsdale.python.org>
Message-ID: <201106040006.11119.victor.stinner@haypocalc.com>

Le vendredi 03 juin 2011 17:28:48, eric.araujo a ?crit :
> +packaging
> +---------
> +
> +:mod:`distutils` has undergone additions and refactoring under a new name,
> +:mod:`packaging`, to allow developers to break backward compatibility.
> +:mod:`distutils` is still provided in the standard library, but users are
> +encouraged to transition to :mod:`packaging`.  For older versions of
> Python, a +backport compatible with 2.4+ and 3.1+ will be made available
> on PyPI under the +name :mod:`distutils2`.

Even for Python 2.4, really? Do you really need to support this old Python 
release?

Victor

From ziade.tarek at gmail.com  Sat Jun  4 00:12:34 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 4 Jun 2011 00:12:34 +0200
Subject: [Python-Dev] Sniffing passwords from PyPI using insecure
	connection
In-Reply-To: <4DE954C5.7080609@v.loewis.de>
References: <BANLkTikfTXNrvBQpJ-_kXHePon7ynAmOGw@mail.gmail.com>
	<is3e5o$qou$1@dough.gmane.org> <4DE5D005.5000001@v.loewis.de>
	<is4mgf$6uh$1@dough.gmane.org>
	<20110601070818.244edc03@neurotica.wooz.org>
	<4DE954C5.7080609@v.loewis.de>
Message-ID: <BANLkTimKVWeXEmqTXjxj-ZsAs19ELbTX1w@mail.gmail.com>

On Fri, Jun 3, 2011 at 11:40 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I followed up on the tracker. ?I'm +0 on adding this to 2.6, but not until
>> after the 2.6.7 release on Friday.
>>
>> How well has this change been tested? ?Are there people for whom this could
>> break things?
>
> As others have pointed out: it would break systems that don't have the
> _ssl module built.

yeah, we would need to fallback to http in that case.

while using https by default is a nice addition, maybe we should also
look at adding a scp-like upload/register command, since the server
has now this ability.


>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
>



-- 
Tarek Ziad? | http://ziade.org

From tseaver at palladion.com  Sat Jun  4 00:29:02 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 03 Jun 2011 18:29:02 -0400
Subject: [Python-Dev] [Python-checkins] cpython: Add NEWS and whatsnew
 entries for the packaging module
In-Reply-To: <201106040006.11119.victor.stinner@haypocalc.com>
References: <E1QSWIq-0002aZ-DD@dinsdale.python.org>
	<201106040006.11119.victor.stinner@haypocalc.com>
Message-ID: <isbn7d$dbq$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/03/2011 06:06 PM, Victor Stinner wrote:
> Le vendredi 03 juin 2011 17:28:48, eric.araujo a ?crit :
>> +packaging
>> +---------
>> +
>> +:mod:`distutils` has undergone additions and refactoring under a new name,
>> +:mod:`packaging`, to allow developers to break backward compatibility.
>> +:mod:`distutils` is still provided in the standard library, but users are
>> +encouraged to transition to :mod:`packaging`.  For older versions of
>> Python, a +backport compatible with 2.4+ and 3.1+ will be made available
>> on PyPI under the +name :mod:`distutils2`.
> 
> Even for Python 2.4, really? Do you really need to support this old Python 

Yes.  Many projects distribute packages to folks still using 2.4.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk3pYC0ACgkQ+gerLs4ltQ638ACghJiU7Ts7D5PrcfA2r1ZoJklW
7zkAn2vJwI1W1N7GVTrpR6/8Lt48ltBz
=xKNv
-----END PGP SIGNATURE-----


From barry at python.org  Sat Jun  4 02:18:31 2011
From: barry at python.org (Barry Warsaw)
Date: Fri, 3 Jun 2011 20:18:31 -0400
Subject: [Python-Dev] Released: Python 2.6.7
Message-ID: <20110603201831.6f488702@neurotica.wooz.org>

Hello Pythoneers and Pythonistas,

I'm happy to announce the final release of Python 2.6.7.

Python 2.6 is in security-fix only mode.  This means that general bug
maintenance has ended, and only critical security issues are being fixed.
We will support Python 2.6 in security-fix only mode until October 2013.
Also, this is a source-only release; no installers for Windows or Mac OS X
will be provided.

Please download the source from:

    http://www.python.org/download/releases/2.6.7/

The NEWS file contains the list of changes since 2.6.6:

    http://www.python.org/download/releases/2.6.7/NEWS.txt

Many thanks go out to the entire Python community for their contributions and
help in making Python 2.6.7 available.

Enjoy,
-Barry
(on behalf of the Python development community)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110603/7524a6e7/attachment.pgp>

From drsalists at gmail.com  Sat Jun  4 02:29:49 2011
From: drsalists at gmail.com (Dan Stromberg)
Date: Fri, 3 Jun 2011 17:29:49 -0700
Subject: [Python-Dev] [Python-checkins] cpython: Add NEWS and whatsnew
 entries for the packaging module
In-Reply-To: <isbn7d$dbq$1@dough.gmane.org>
References: <E1QSWIq-0002aZ-DD@dinsdale.python.org>
	<201106040006.11119.victor.stinner@haypocalc.com>
	<isbn7d$dbq$1@dough.gmane.org>
Message-ID: <BANLkTimZv91mStgTVH0ALf5VwuYxh3rm1w@mail.gmail.com>

On Fri, Jun 3, 2011 at 3:29 PM, Tres Seaver <tseaver at palladion.com> wrote:

> > Even for Python 2.4, really? Do you really need to support this old
> Python
>
>
> Yes.  Many projects distribute packages to folks still using 2.4.
>

Supporting detail: I recently installed the latest CentOS, 5.6, and found
that it still Ships with CPython 2.4.

I installed a CPython 3.2 in /usr/local almost immediately, but I can see
how some might want 2.4 support yet.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110603/a4e61424/attachment.html>

From ncoghlan at gmail.com  Sat Jun  4 07:54:17 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 4 Jun 2011 15:54:17 +1000
Subject: [Python-Dev] [Python-checkins] cpython: Add NEWS and whatsnew
 entries for the packaging module
In-Reply-To: <BANLkTimZv91mStgTVH0ALf5VwuYxh3rm1w@mail.gmail.com>
References: <E1QSWIq-0002aZ-DD@dinsdale.python.org>
	<201106040006.11119.victor.stinner@haypocalc.com>
	<isbn7d$dbq$1@dough.gmane.org>
	<BANLkTimZv91mStgTVH0ALf5VwuYxh3rm1w@mail.gmail.com>
Message-ID: <BANLkTimiD6gkVodVpAnpGeDU_BwfqafJ4Q@mail.gmail.com>

On Sat, Jun 4, 2011 at 10:29 AM, Dan Stromberg <drsalists at gmail.com> wrote:
> Supporting detail: I recently installed the latest CentOS, 5.6, and found
> that it still Ships with CPython 2.4.
>
> I installed a CPython 3.2 in /usr/local almost immediately, but I can see
> how some might want 2.4 support yet.

Yeah, this came up at Brispy last week. Until CentOS 6 comes out and
hosting providers based on CentOS make that version available to their
customers, quite a few people are going to want 2.4 support. That's
likely to take a while, since the last note from the CentOS folks was
a month ago [1], and essentially just said "soon". Once it happens,
CentOS 6 users will be able to join RHEL6 users on 2.6.2.

[1] https://www.centos.org/modules/newbb/viewtopic.php?topic_id=25878&forum=53#12

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From solipsis at pitrou.net  Sat Jun  4 20:50:26 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 4 Jun 2011 20:50:26 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <4DE2643D.3070906@v.loewis.de>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
Message-ID: <20110604205026.374c0019@pitrou.net>


Hello,

On Sun, 29 May 2011 17:20:29 +0200
"Martin v. L?wis" <martin at v.loewis.de> wrote:
> > I would like to suggest that we remove the socket HOWTO (currently at
> > http://docs.python.org/dev/howto/sockets.html)
> 
> -1. I think there should be a Python-oriented introduction to sockets.
> You may have complaints about the specific wording of the text, but
> please understand that these are probably irrelevant to most
> first-time readers of this text. My observation is that people actually
> don't read the text that much, but instead try to imitate the examples.

So what you're saying is that the text is mostly useless (or at least
quite dispensable), but you think it's fine that people waste their
time trying to read it?

Some of the people reading our docs are not fluent English readers, and
it can be quite an effort for them to read some big chunk of text which
will be ultimately pointless.

> So if the examples are good (and I think they are, mostly), it's of
> minor relevance whether the text makes all sense the first time.

I'm not sure why the examples are good (for example, modern client
code should probably use create_connection() with a host name, not
connect()).

Also, really, to socket beginners, I think the primary advice should
be: first try to find some high-level library that does the dirty work
for you (for example some protocol-specific lib on the client side, or
something like Twisted or asyncore on the server side). Not "hey,
here's how you write a threaded server in 4 lines of code, and wow,
look, you can also write non-blocking code using select() too!".

> It's not important
> to first-time readers to actually understand that, and the wording
> explicitly tells them that they don't need to understand. It says
> "there is more stuff, and you won't need it, and the stuff you need
> is called INET and STREAM".

Well... in a couple of months, someone will tell them their code doesn't
support IPv6 and they'll be lost.

> > what's "select"?)
> 
> It's well explained in the section Non-blocking Sockets, isn't it?

I don't think it explains well how a non-blocking socket works.
It's very opinionated and has little useful technical content. 
EAGAIN and EWOULDBLOCK are not even mentioned!

> It's a HOWTO - of course it has advise without justification.

Well, I think that's bad. When we give advice to users, we should
explain the motivation of the advice given. Otherwise we convey the
impression that there's some magic that people shouldn't try to
understand.

> > what is a "nasty death" and how is that supposed to
> > happen? couldn't the author have put a 3-line example to demonstrate
> > this supposed drawback and how it manifests?).
> 
> It may well be that the author didn't fully understand the problem
> when writing the text, so I wouldn't mind removing this specific
> paragraph.

+1. When reading it I get the idea that the OS might kill sockets in
my back, while in reality the only way a EBADF can happen is if I
explicitly close the socket - i.e. a programming error on my part.

> I'd drop the entire "Performance" section - there is much more
> to be said about socket performance than a few paragraphs of text,
> and for the target audience, performance is probably no concern.

+1 :)

Thank you

Antoine.

From nyamatongwe at gmail.com  Sun Jun  5 00:58:00 2011
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Sun, 5 Jun 2011 08:58:00 +1000
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <20110604205026.374c0019@pitrou.net>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net>
Message-ID: <BANLkTik0ZUtUN-kNFrSqXkfHVr9Fc28uWw@mail.gmail.com>

Antoine Pitrou:

> So what you're saying is that the text is mostly useless (or at least
> quite dispensable), but you think it's fine that people waste their
> time trying to read it?

   I found it useful when starting to write socket code. Later on I
learnt more but, as an introduction, this document was great. It is
written in an approachable manner and doesn't spend time on details
unimportant to initial understanding.

   Neil

From benjamin at python.org  Sun Jun  5 03:45:31 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 4 Jun 2011 20:45:31 -0500
Subject: [Python-Dev] keyword-only arguments and varags
Message-ID: <BANLkTinmUg9eTa6+ejdKysWa1qieX=nHbA@mail.gmail.com>

Currently,

def f(*, kw, **kwargs): pass

is valid syntax, but

def f(*args, *, kw): pass

is not.

I don't see any mention of it in the PEP but perhaps there is a good
reason this isn't allowed. It seems to be perfectly well-defined to
me.

-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Sun Jun  5 07:58:09 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 5 Jun 2011 15:58:09 +1000
Subject: [Python-Dev] keyword-only arguments and varags
In-Reply-To: <BANLkTinmUg9eTa6+ejdKysWa1qieX=nHbA@mail.gmail.com>
References: <BANLkTinmUg9eTa6+ejdKysWa1qieX=nHbA@mail.gmail.com>
Message-ID: <BANLkTi=vtJ3LoKkox25baEM4jKGizn48+A@mail.gmail.com>

On Sun, Jun 5, 2011 at 11:45 AM, Benjamin Peterson <benjamin at python.org> wrote:
> Currently,
>
> def f(*, kw, **kwargs): pass
>
> is valid syntax, but
>
> def f(*args, *, kw): pass
>
> is not.
>
> I don't see any mention of it in the PEP but perhaps there is a good
> reason this isn't allowed. It seems to be perfectly well-defined to
> me.

Really? There's two single-stars there. One says "accept arbitrary
positional arguments and place them in a tuple named args", the second
says "don't accept any more positional args". You can't have it both
ways, so the compiler complains.

The following works fine to mix keyword-only arguments with arbitrary
positional arguments, so I don't see a problem:

def f(*args, kw): pass

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From martin at v.loewis.de  Sun Jun  5 08:32:38 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 05 Jun 2011 08:32:38 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <20110604205026.374c0019@pitrou.net>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net>
Message-ID: <4DEB2306.5080206@v.loewis.de>

>> -1. I think there should be a Python-oriented introduction to sockets.
>> You may have complaints about the specific wording of the text, but
>> please understand that these are probably irrelevant to most
>> first-time readers of this text. My observation is that people actually
>> don't read the text that much, but instead try to imitate the examples.
> 
> So what you're saying is that the text is mostly useless (or at least
> quite dispensable), but you think it's fine that people waste their
> time trying to read it?

No, that's not what I said. I said the people actually *don't* read
the text, so they won't waste time with it. They only glance at the
text, enough to understand the examples.

> Some of the people reading our docs are not fluent English readers, and
> it can be quite an effort for them to read some big chunk of text which
> will be ultimately pointless.

You completely misunderstood. I didn't say that the reading the text is
pointless. I said that people don't read this text, nor any software
documentation, in particular when they are not fluent in English.

>> So if the examples are good (and I think they are, mostly), it's of
>> minor relevance whether the text makes all sense the first time.
> 
> I'm not sure why the examples are good (for example, modern client
> code should probably use create_connection() with a host name, not
> connect()).

I disagree. create_connection is an advanced function - you shouldn't
be using it unless you know what it is doing. As a socket tutorial,
people do have to know about connect.

> Also, really, to socket beginners, I think the primary advice should
> be: first try to find some high-level library that does the dirty work
> for you (for example some protocol-specific lib on the client side, or
> something like Twisted or asyncore on the server side).

No no no no no. Absolutely not.
a) telling people who want to learn sockets "don't learn sockets,
   learn some higher-level library" is besides the point. What do
   you tell them when they did that, and now come back to learn
   sockets?
b) telling people to use Twisted or asyncore on the server side
   if they are new to sockets is bad advice. People *first* have
   to understand sockets, and *then* can use these libraries
   and frameworks. Those libraries aren't made to be black boxes
   that work even if you don't know how - you *have* to know how
   they work inside, or else you can't productively use them.

> Not "hey,
> here's how you write a threaded server in 4 lines of code, and wow,
> look, you can also write non-blocking code using select() too!".

I'd happily kill the entire non-blocking discussion from the tutorial
if it hurts you as much as it hurts me. I wish this non-blocking stuff
never went into Python.

> Well... in a couple of months, someone will tell them their code doesn't
> support IPv6 and they'll be lost.

No. In a couple of months, they'll understand sockets much better, so
they'll be able to support IPv6 easily.

>> It's a HOWTO - of course it has advise without justification.
> 
> Well, I think that's bad. When we give advice to users, we should
> explain the motivation of the advice given. Otherwise we convey the
> impression that there's some magic that people shouldn't try to
> understand.

It's not that bad. Please reconsider. People do get a lot of advise
in their lives that isn't motivated down to the root cause, and accept
advise from authority. Only if they understand what it does, they ask
why.

Regards,
Martin

From victor.stinner at haypocalc.com  Sun Jun  5 11:38:00 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 5 Jun 2011 11:38:00 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.1): Do not add txt
	files twice.
In-Reply-To: <E1QT9CA-00030t-WD@dinsdale.python.org>
References: <E1QT9CA-00030t-WD@dinsdale.python.org>
Message-ID: <201106051138.00689.victor.stinner@haypocalc.com>

I added the "if dir=='cjkencodings':" to msi.py, based on tests for other 
subdirectories in Lib/tests/. Can you explain me why cjkencodings should not 
have a special case? The fix should maybe be ported to 3.2, 3.3 and 2.7.

Victor

Le dimanche 05 juin 2011 11:00:30, martin.v.loewis a ?crit :
> http://hg.python.org/cpython/rev/df7b9c5d6ba1
> changeset:   70643:df7b9c5d6ba1
> branch:      3.1
> parent:      70571:0639e630426c
> user:        Martin v. L?wis <martin at v.loewis.de>
> date:        Sun Jun 05 10:55:57 2011 +0200
> summary:
>   Do not add txt files twice.
> 
> files:
>   Tools/msi/msi.py |  2 --
>   1 files changed, 0 insertions(+), 2 deletions(-)
> 
> 
> diff --git a/Tools/msi/msi.py b/Tools/msi/msi.py
> --- a/Tools/msi/msi.py
> +++ b/Tools/msi/msi.py
> @@ -1021,8 +1021,6 @@
>              lib.add_file("zipdir.zip")
>          if dir=='tests' and parent.physical=='distutils':
>              lib.add_file("Setup.sample")
> -        if dir=='cjkencodings':
> -            lib.glob("*.txt")
>          if dir=='decimaltestdata':
>              lib.glob("*.decTest")
>          if dir=='output':

From solipsis at pitrou.net  Sun Jun  5 12:33:13 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 5 Jun 2011 12:33:13 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <4DEB2306.5080206@v.loewis.de>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
Message-ID: <20110605123313.64a4a720@pitrou.net>

On Sun, 05 Jun 2011 08:32:38 +0200
"Martin v. L?wis" <martin at v.loewis.de> wrote:
> >> -1. I think there should be a Python-oriented introduction to sockets.
> >> You may have complaints about the specific wording of the text, but
> >> please understand that these are probably irrelevant to most
> >> first-time readers of this text. My observation is that people actually
> >> don't read the text that much, but instead try to imitate the examples.
> > 
> > So what you're saying is that the text is mostly useless (or at least
> > quite dispensable), but you think it's fine that people waste their
> > time trying to read it?
> 
> No, that's not what I said. I said the people actually *don't* read
> the text, so they won't waste time with it. They only glance at the
> text, enough to understand the examples.

I'm sorry, that sounds like a very outlandish argument to make.
Did you run a user survey?

If people only "glance at the text", then what is the text for?
Why not kill the text and rename the page "socket examples" so that
there is no misunderstanding and so that we don't waste time
trying to maintain (and argue about) it?

> > I'm not sure why the examples are good (for example, modern client
> > code should probably use create_connection() with a host name, not
> > connect()).
> 
> I disagree. create_connection is an advanced function - you shouldn't
> be using it unless you know what it is doing.

Can you explain? I would certainly use it myself, and I don't
understand how it's "advanced". It's simply higher-level.

Actually, we've been actually replacing uses of connect() with
create_connection() in various parts of the stdlib, so that our
client modules get IPv6-compatible.

> No no no no no. Absolutely not.
> a) telling people who want to learn sockets "don't learn sockets,
>    learn some higher-level library" is besides the point. What do
>    you tell them when they did that, and now come back to learn
>    sockets?

You said yourself that the HOWTO doesn't claim to explain sockets, so
how can you make such a point now? If people want to learn sockets for
real, the HOWTO is hopeless for them.

> I'd happily kill the entire non-blocking discussion from the tutorial
> if it hurts you as much as it hurts me.

+1.

Regards

Antoine.

From aahz at pythoncraft.com  Sun Jun  5 17:20:25 2011
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 5 Jun 2011 08:20:25 -0700
Subject: [Python-Dev] FWD: gpg keys have problems
Message-ID: <20110605152025.GA25276@panix.com>

I'm not currently reading python-dev, dunno if this has come up before:

----- Forwarded message from "Michael J. Dinneen" <mjd at cs.auckland.ac.nz> -----

> Date: Sun, 05 Jun 2011 19:33:04 +1200
> From: "Michael J. Dinneen" <mjd at cs.auckland.ac.nz>
> To: webmaster at python.org
> Subject: gpg keys have problems
> Organization: University of Auckland
> 
> >From your python download page you need to update the public keys to not use the faulty MD5 digest algorithm.
> (see the link listed below)
> 
> $ gpg --import pubkeys.txt
> gpg: key 6A45C816: public key "Anthony Baxter <anthony at interlink.com.au>" imported
> gpg: WARNING: digest algorithm MD5 is deprecated
> gpg: please see http://www.gnupg.org/faq/weak-digest-algos.html for more information
> gpg: key ED9D77D5: public key "Barry A. Warsaw <barry at warsaw.us>" imported
> gpg: Total number processed: 2
> gpg:               imported: 2  (RSA: 1)
> gpg: no ultimately trusted keys found
> 

----- End forwarded message -----

-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Looking back over the years, after I learned Python I realized that I
never really had enjoyed programming before.

From benjamin at python.org  Sun Jun  5 18:31:44 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 5 Jun 2011 11:31:44 -0500
Subject: [Python-Dev] keyword-only arguments and varags
In-Reply-To: <BANLkTi=vtJ3LoKkox25baEM4jKGizn48+A@mail.gmail.com>
References: <BANLkTinmUg9eTa6+ejdKysWa1qieX=nHbA@mail.gmail.com>
	<BANLkTi=vtJ3LoKkox25baEM4jKGizn48+A@mail.gmail.com>
Message-ID: <BANLkTimBnhVaokrxHaPs30O6P0SCwybiDw@mail.gmail.com>

2011/6/5 Nick Coghlan <ncoghlan at gmail.com>:
> On Sun, Jun 5, 2011 at 11:45 AM, Benjamin Peterson <benjamin at python.org> wrote:
>> Currently,
>>
>> def f(*, kw, **kwargs): pass
>>
>> is valid syntax, but
>>
>> def f(*args, *, kw): pass
>>
>> is not.
>>
>> I don't see any mention of it in the PEP but perhaps there is a good
>> reason this isn't allowed. It seems to be perfectly well-defined to
>> me.
>
> Really? There's two single-stars there. One says "accept arbitrary
> positional arguments and place them in a tuple named args", the second
> says "don't accept any more positional args". You can't have it both
> ways, so the compiler complains.

Thank you. More proof I shouldn't write emails after 22:00 local time.

>
> The following works fine to mix keyword-only arguments with arbitrary
> positional arguments, so I don't see a problem:
>
> def f(*args, kw): pass

Move along, nothing to see here...



-- 
Regards,
Benjamin

From glyph at twistedmatrix.com  Sun Jun  5 22:28:10 2011
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Sun, 5 Jun 2011 13:28:10 -0700
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <4DEB2306.5080206@v.loewis.de>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
Message-ID: <1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>

On Jun 4, 2011, at 11:32 PM, Martin v. L?wis wrote:

> b) telling people to use Twisted or asyncore on the server side
>   if they are new to sockets is bad advice. People *first* have
>   to understand sockets, and *then* can use these libraries
>   and frameworks. Those libraries aren't made to be black boxes
>   that work even if you don't know how - you *have* to know how
>   they work inside, or else you can't productively use them.


First, Twisted doesn't always use the BSD sockets API; the Windows IOCP reactor, especially, starts off with the socket() function, but things go off in a different direction pretty quickly from there.  So it's perfectly fine to introduce yourself to networking via Twisted, and many users have done just that.  If you're using it idiomatically, you should never encounter a socket object or file descriptor poking through the API anywhere.  Asyncore is different: you do need to know how sockets work in order to use it, because you're expected to call .send() and .recv() yourself.  (And, in my opinion, this is a serious design flaw, for reasons which will hopefully be elucidated in the PEP that Laurens is now writing.)

Second, it makes me a little sad that it appears to be folk wisdom that Twisted is only for servers.  A lot of work has gone into making it equally appropriate for clients.  This is especially true if your client has a GUI, where Twisted is often better than a protocol-specific library, which may either be blocking or have its own ad-hoc event loop.

I don't have an opinion on the socket HOWTO per se, only on the possibility of linking to Twisted as an alternate implementation mechanism.  It really would be better to say "go use Twisted rather than reading any of the following" than "read the following, which will help you understand Twisted".


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110605/31700ba7/attachment.html>

From raymond.hettinger at gmail.com  Sun Jun  5 23:13:07 2011
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sun, 5 Jun 2011 14:13:07 -0700
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
Message-ID: <857C8A06-E922-428A-BBE3-09FB63640C05@gmail.com>



> On Jun 4, 2011, at 11:32 PM, Martin v. L?wis wrote:
> 
>> b) telling people to use Twisted or asyncore on the server side
>>   if they are new to sockets is bad advice. People *first* have
>>   to understand sockets, and *then* can use these libraries
>>   and frameworks. Those libraries aren't made to be black boxes
>>   that work even if you don't know how - you *have* to know how
>>   they work inside, or else you can't productively use them.

It would be fine to have a separate networking howto guide that covers
libraries, frameworks, and high level APIs, but I agree with Martin that
the Socket HOWTO needs to cover sockets.  That's what a person expects
to learn when they click on the Socket HOWTO link.

Raymond

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110605/d4b4edde/attachment.html>

From martin at v.loewis.de  Mon Jun  6 00:22:11 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 06 Jun 2011 00:22:11 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <20110605123313.64a4a720@pitrou.net>
References: <20110521170725.51eab5f9@pitrou.net>
	<4DE2643D.3070906@v.loewis.de>	<20110604205026.374c0019@pitrou.net>
	<4DEB2306.5080206@v.loewis.de> <20110605123313.64a4a720@pitrou.net>
Message-ID: <4DEC0193.5080208@v.loewis.de>

>>> I'm not sure why the examples are good (for example, modern client
>>> code should probably use create_connection() with a host name, not
>>> connect()).
>>
>> I disagree. create_connection is an advanced function - you shouldn't
>> be using it unless you know what it is doing.
> 
> Can you explain? I would certainly use it myself, and I don't
> understand how it's "advanced". It's simply higher-level.

It uses getaddrinfo, which might return multiple addresses, which
are then tried in sequence. So even though it's called
"create_connection", it may actually attempt to create multiple
connections. As a consequence, it may wait some time for one connection
to complete, and then succeed on a different address.

These phenomena can only be understood when you know what it is
actually doing.

> Actually, we've been actually replacing uses of connect() with
> create_connection() in various parts of the stdlib, so that our
> client modules get IPv6-compatible.

And that's fine - the people making this changes most certainly
where capable of using advanced API.

>> No no no no no. Absolutely not.
>> a) telling people who want to learn sockets "don't learn sockets,
>>    learn some higher-level library" is besides the point. What do
>>    you tell them when they did that, and now come back to learn
>>    sockets?
> 
> You said yourself that the HOWTO doesn't claim to explain sockets

Did I say that? If so, I didn't mean to. It explains how to use the
socket API.

Regards,
Martin

From martin at v.loewis.de  Mon Jun  6 00:35:54 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 06 Jun 2011 00:35:54 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
References: <20110521170725.51eab5f9@pitrou.net>
	<4DE2643D.3070906@v.loewis.de>	<20110604205026.374c0019@pitrou.net>
	<4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
Message-ID: <4DEC04CA.1060608@v.loewis.de>

> First, Twisted doesn't always use the BSD sockets API; the Windows IOCP
> reactor, especially, starts off with the socket() function, but things
> go off in a different direction pretty quickly from there.

Hmm. Are you saying it doesn't use listen, connect, bind, send, recv?
To me, that's the core of BSD sockets. I can understand it doesn't
use select(2).

> So it's
> perfectly fine to introduce yourself to networking via Twisted, and many
> users have done just that.  If you're using it idiomatically, you should
> never encounter a socket object or file descriptor poking through the
> API anywhere.

And that's all fine. I still claim that you have to *understand*
sockets in order to use it properly. By this, I mean stuff like
"what is a TCP connection? how is it established?", "how is UDP
different from TCP?", "when data arrives, what layers of software
does it go through?", "what is a port number?", etc.

> Second, it makes me a little sad that it appears to be folk wisdom that
> Twisted is only for servers.  A lot of work has gone into making it
> equally appropriate for clients.  This is especially true if your client
> has a GUI, where Twisted is often better than a protocol-specific
> library, which may either be blocking or have its own ad-hoc event loop.

I think that's because many of the problems that Twisted solves don't
exist in many of the client applications - in particular, you often
don't have many simultaneous connections. GUI apps may be the
interesting special case, but I expect that people dealing with these
rather use separate threads.

> I don't have an opinion on the socket HOWTO per se, only on the
> possibility of linking to Twisted as an alternate implementation
> mechanism.  It really would be better to say "go use Twisted rather than
> reading any of the following" than "read the following, which will help
> you understand Twisted".

Wouldn't you agree that Twisted is very difficult to learn, and thus
much heavier than sockets? And I don't blame the Twisted API for that,
but rather the mental model of overlapping activities that people have
severe problems with.

Regards,
Martin

From exarkun at twistedmatrix.com  Mon Jun  6 02:40:35 2011
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Mon, 06 Jun 2011 00:40:35 -0000
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <4DEC04CA.1060608@v.loewis.de>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
	<4DEC04CA.1060608@v.loewis.de>
Message-ID: <20110606004035.3983.961114140.divmod.xquotient.546@localhost.localdomain>

On 5 Jun, 10:35 pm, martin at v.loewis.de wrote:
>>First, Twisted doesn't always use the BSD sockets API; the Windows 
>>IOCP
>>reactor, especially, starts off with the socket() function, but things
>>go off in a different direction pretty quickly from there.
>
>Hmm. Are you saying it doesn't use listen, connect, bind, send, recv?
>To me, that's the core of BSD sockets. I can understand it doesn't
>use select(2).

Yes, that's correct.  Those aren't the best APIs to use on Windows, so 
they aren't necessarily used on Windows.
>>So it's
>>perfectly fine to introduce yourself to networking via Twisted, and 
>>many
>>users have done just that.  If you're using it idiomatically, you 
>>should
>>never encounter a socket object or file descriptor poking through the
>>API anywhere.
>
>And that's all fine. I still claim that you have to *understand*
>sockets in order to use it properly. By this, I mean stuff like
>"what is a TCP connection? how is it established?", "how is UDP
>different from TCP?", "when data arrives, what layers of software
>does it go through?", "what is a port number?", etc.

These may be good things to understand.  The current socket howto 
doesn't explain them, though.
>>Second, it makes me a little sad that it appears to be folk wisdom 
>>that
>>Twisted is only for servers.  A lot of work has gone into making it
>>equally appropriate for clients.  This is especially true if your 
>>client
>>has a GUI, where Twisted is often better than a protocol-specific
>>library, which may either be blocking or have its own ad-hoc event 
>>loop.
>
>I think that's because many of the problems that Twisted solves don't
>exist in many of the client applications - in particular, you often
>don't have many simultaneous connections. GUI apps may be the
>interesting special case, but I expect that people dealing with these
>rather use separate threads.

On the contrary, many of the problems do exist in client applications 
(every time I have to use virt-manager I want to throw it out a window).

Some people certainly would rather use threading, but that doesn't say 
anything about whether Twisted solves problems relevant to clients, only 
about the fact that a lot of people like to use threads.
>>I don't have an opinion on the socket HOWTO per se, only on the
>>possibility of linking to Twisted as an alternate implementation
>>mechanism.  It really would be better to say "go use Twisted rather 
>>than
>>reading any of the following" than "read the following, which will 
>>help
>>you understand Twisted".
>
>Wouldn't you agree that Twisted is very difficult to learn, and thus
>much heavier than sockets? And I don't blame the Twisted API for that,
>but rather the mental model of overlapping activities that people have
>severe problems with.

This discussion has a significant problem, in taking "Twisted" as a 
monolithic all-or-nothing entity.  Restricting the scope to merely the 
lowest-level socket replacement APIs - ie, the bare TCP, UDP, etc 
functionality - no, Twisted is not very difficult to learn.  Expanding 
the scope to include the higher level functionality, it is much easier 
to learn than reimplementing line parsing and concurrency and so forth.

However, does that really have anything to do with improving the socket 
howto?  The Python documentation can include a clear explanation of what 
functionality the socket module provides - without forcing you to read 
Stevens _or_ use Twisted, but it can still refer you to both, since it 
is very likely that you'll need at least one of them in addition to the 
socket module.

Jean-Paul

From nadeem.vawda at gmail.com  Mon Jun  6 08:06:36 2011
From: nadeem.vawda at gmail.com (Nadeem Vawda)
Date: Mon, 6 Jun 2011 08:06:36 +0200
Subject: [Python-Dev] Add LRUCache class to stdlib
Message-ID: <BANLkTiknSKEVPXNPwAJxAk9GUPrjZCjbNg@mail.gmail.com>

I would like to propose adding a class to the stdlib to provide a more flexible
LRU caching mechanism. As things stand, the functools.lru_cache() decorator is
fine for memoization of pure functions, but does not accommodate situations
where cache entries become stale and must be invalidated/updated (e.g.
filecmp.cmp(); cf. issue #11802).

I was thinking it would be a good idea to factor out the the replacement code
into a separate class that could then be reused by code for which the
lru_cache() decorator is not applicable. The implementation of lru_cache()
itself would also become considerably simpler.

Implementation:

    class LRUCache:
        """Least-recently-used cache class.

        See:  http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used

        """

        def __init__(self, maxsize):
            """Initialize an LRUCache.

            If *maxsize* is None, the LRU replacement mechanism is disabled,
            and the cache can grow without bound.

            """
            if maxsize is None:
                self.cache = dict()
            else:
                self.cache = OrderedDict()
            self.lock = Lock()
            self.hits = self.misses = 0
            self.maxsize = maxsize

        def __getitem__(self, key):
            with self.lock:
                try:
                    value = self.cache[key]
                except KeyError:
                    self.misses += 1
                    raise
                else:
                    self.hits += 1
                    if self.maxsize is not None:
                        self.cache.move_to_end(key)
                    return value

        def __setitem__(self, key, value):
            with self.lock:
                self.cache[key] = value
                if self.maxsize is not None and len(self.cache) > self.maxsize:
                    self.cache.popitem(0)

        def info(self):
            """Report cache statistics"""
            with self.lock:
                return _CacheInfo(self.hits, self.misses,
                                  self.maxsize, len(self.cache))

I'm not sure where this class should go; it would be convenient to just stick
it in functools along with lru_cache(), but perhaps the collections package
would be more appropriate?

Cheers,
Nadeem

From solipsis at pitrou.net  Mon Jun  6 09:17:20 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 6 Jun 2011 09:17:20 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <4DEC0193.5080208@v.loewis.de>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<20110605123313.64a4a720@pitrou.net> <4DEC0193.5080208@v.loewis.de>
Message-ID: <20110606091720.0ddffff1@pitrou.net>

On Mon, 06 Jun 2011 00:22:11 +0200
"Martin v. L?wis" <martin at v.loewis.de> wrote:
> >>> I'm not sure why the examples are good (for example, modern client
> >>> code should probably use create_connection() with a host name, not
> >>> connect()).
> >>
> >> I disagree. create_connection is an advanced function - you shouldn't
> >> be using it unless you know what it is doing.
> > 
> > Can you explain? I would certainly use it myself, and I don't
> > understand how it's "advanced". It's simply higher-level.
> 
> It uses getaddrinfo, which might return multiple addresses, which
> are then tried in sequence. So even though it's called
> "create_connection", it may actually attempt to create multiple
> connections. As a consequence, it may wait some time for one connection
> to complete, and then succeed on a different address.
> 
> These phenomena can only be understood when you know what it is
> actually doing.

So what? You can say the exact same thing about every API under the sun.
Yet the sockets HOWTO *doesn't* explain what the socket APIs are
actually doing.


From g.brandl at gmx.net  Mon Jun  6 09:50:54 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 06 Jun 2011 09:50:54 +0200
Subject: [Python-Dev] cpython: always clear parser error
In-Reply-To: <E1QTQGA-00073i-6v@dinsdale.python.org>
References: <E1QTQGA-00073i-6v@dinsdale.python.org>
Message-ID: <isi0st$2sa$1@dough.gmane.org>

On 06/06/11 05:13, benjamin.peterson wrote:
> http://hg.python.org/cpython/rev/9faa9270457e
> changeset:   70654:9faa9270457e
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Sun Jun 05 22:14:05 2011 -0500
> summary:
>   always clear parser error
> 
> files:
>   Modules/parsermodule.c |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)

This looks like a fixup of rev 3ffd8dea77bf; it would be nice to say
so in the commit message: the current message is not obvious at all.

Georg


From stephen at xemacs.org  Mon Jun  6 09:52:31 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 6 Jun 2011 16:52:31 +0900
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <20110606004035.3983.961114140.divmod.xquotient.546@localhost.localdomain>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
	<4DEC04CA.1060608@v.loewis.de>
	<20110606004035.3983.961114140.divmod.xquotient.546@localhost.localdomain>
Message-ID: <19948.34623.477105.907414@uwakimon.sk.tsukuba.ac.jp>

exarkun at twistedmatrix.com writes:

 > However, does that really have anything to do with improving the socket 
 > howto?

Thank you!

 > The Python documentation can include a clear explanation of what 
 > functionality the socket module provides - without forcing you to read 
 > Stevens _or_ use Twisted, but it can still refer you to both, since it 
 > is very likely that you'll need at least one of them in addition to the 
 > socket module.

+1

I suggest s/very likely/likely/, though.


From stephen at xemacs.org  Mon Jun  6 10:01:15 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 6 Jun 2011 17:01:15 +0900
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <20110605123313.64a4a720@pitrou.net>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<20110605123313.64a4a720@pitrou.net>
Message-ID: <19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > Did you run a user survey?

Martin undoubtedly has a lot of experience with users, and it's quite
reasonable for him to express his opinions based on that informal
sample, yes.

The issue here is the difference between existential and universal
quantifiers.  Martin's arguments are not inconsistent.  They simply
acknowledge the existence of subsamples of users of the same document
with different needs and/or approaches to reading the document.  He
does not and has never claimed that all of his arguments apply to all
of the potential readers.

You might question whether the same document should serve both the
"cargo cult the examples" group and the "read the fine print" group.
That's a valid question, but here my feeling is that the answer is
"yes".  I very often "cargo cult" my first program, then go back to
the fine print and experiment by gradually changing that program to
test my understanding of the detailed explanations.  It is often
easiest to use the same document for both purposes because I already
know where it is and the quality of the writing.


From solipsis at pitrou.net  Mon Jun  6 10:11:21 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 06 Jun 2011 10:11:21 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>
References: <20110521170725.51eab5f9@pitrou.net>
	<4DE2643D.3070906@v.loewis.de> <20110604205026.374c0019@pitrou.net>
	<4DEB2306.5080206@v.loewis.de> <20110605123313.64a4a720@pitrou.net>
	<19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1307347881.3506.1.camel@localhost.localdomain>

Le lundi 06 juin 2011 ? 17:01 +0900, Stephen J. Turnbull a ?crit :
> You might question whether the same document should serve both the
> "cargo cult the examples" group and the "read the fine print" group.
> That's a valid question, but here my feeling is that the answer is
> "yes".

So did you read the discussion before posting? The sockets HOWTO
*doesn't* serve both groups.
Actually, I would argue that it serves neither of them.



From martin at v.loewis.de  Mon Jun  6 10:23:18 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 06 Jun 2011 10:23:18 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>
References: <20110521170725.51eab5f9@pitrou.net>
	<4DE2643D.3070906@v.loewis.de>	<20110604205026.374c0019@pitrou.net>
	<4DEB2306.5080206@v.loewis.de>	<20110605123313.64a4a720@pitrou.net>
	<19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4DEC8E76.5070109@v.loewis.de>

>  > Did you run a user survey?
> 
> Martin undoubtedly has a lot of experience with users, and it's quite
> reasonable for him to express his opinions based on that informal
> sample, yes.

In particular, this is collected experience from interaction with
students learning Python, or other languages. When they try to solve
a problem, they don't read specification-style documentation. Instead
they look for examples that they can imitate.

[I notice that you (Stephen) also confirmed this from your own
 experience]

> The issue here is the difference between existential and universal
> quantifiers.  Martin's arguments are not inconsistent.  They simply
> acknowledge the existence of subsamples of users of the same document
> with different needs and/or approaches to reading the document.  He
> does not and has never claimed that all of his arguments apply to all
> of the potential readers.

Exactly so. I'd like to settle this discussion based on the anecdotal
report of several users on this list that they considered the
tutorial useful.

> You might question whether the same document should serve both the
> "cargo cult the examples" group and the "read the fine print" group.
> That's a valid question, but here my feeling is that the answer is
> "yes".  I very often "cargo cult" my first program, then go back to
> the fine print and experiment by gradually changing that program to
> test my understanding of the detailed explanations.  It is often
> easiest to use the same document for both purposes because I already
> know where it is and the quality of the writing.

In that spirit, I'd be in favor of removing outright errors from the
document, and overly subjective and argumentative passages. Other
than that, I still think its fine as it stands.

Regards,
Martin

From martin at v.loewis.de  Mon Jun  6 10:33:14 2011
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 06 Jun 2011 10:33:14 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <1307347881.3506.1.camel@localhost.localdomain>
References: <20110521170725.51eab5f9@pitrou.net>	<4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net>	<4DEB2306.5080206@v.loewis.de>
	<20110605123313.64a4a720@pitrou.net>	<19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>
	<1307347881.3506.1.camel@localhost.localdomain>
Message-ID: <4DEC90CA.8070604@v.loewis.de>

Am 06.06.2011 10:11, schrieb Antoine Pitrou:
> Le lundi 06 juin 2011 ? 17:01 +0900, Stephen J. Turnbull a ?crit :
>> You might question whether the same document should serve both the
>> "cargo cult the examples" group and the "read the fine print" group.
>> That's a valid question, but here my feeling is that the answer is
>> "yes".
> 
> So did you read the discussion before posting? The sockets HOWTO
> *doesn't* serve both groups.
> Actually, I would argue that it serves neither of them.

How can you make such claims when several people have indicated that
the howto *actually* helped them?

Please let this rest.

Regards,
Martin

From solipsis at pitrou.net  Mon Jun  6 12:13:40 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 6 Jun 2011 12:13:40 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <4DEC90CA.8070604@v.loewis.de>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<20110605123313.64a4a720@pitrou.net>
	<19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>
	<1307347881.3506.1.camel@localhost.localdomain>
	<4DEC90CA.8070604@v.loewis.de>
Message-ID: <20110606121340.0f7a3464@pitrou.net>

On Mon, 06 Jun 2011 10:33:14 +0200
"Martin v. L?wis" <martin at v.loewis.de> wrote:
> Am 06.06.2011 10:11, schrieb Antoine Pitrou:
> > Le lundi 06 juin 2011 ? 17:01 +0900, Stephen J. Turnbull a ?crit :
> >> You might question whether the same document should serve both the
> >> "cargo cult the examples" group and the "read the fine print" group.
> >> That's a valid question, but here my feeling is that the answer is
> >> "yes".
> > 
> > So did you read the discussion before posting? The sockets HOWTO
> > *doesn't* serve both groups.
> > Actually, I would argue that it serves neither of them.
> 
> How can you make such claims when several people have indicated that
> the howto *actually* helped them?

The point here is that the examples in that document are very poor
(the only substantial example actually duplicates existing
functionality - in a sub-optimal manner - without even mentioning the
existence of said functionality), and the technical explanations are
nearly non-existent. So I'll happy stand by my claims.

The Python documentation isn't meant to host any potentially helpful
document, however flawed. We have the Internet for that.

Regards

Antoine.

From victor.stinner at haypocalc.com  Mon Jun  6 12:29:46 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 6 Jun 2011 12:29:46 +0200
Subject: [Python-Dev] Buildbots and regrtest timeout
Message-ID: <201106061229.46577.victor.stinner@haypocalc.com>

Hi,

Stephan Krah asked me to change how the default timeout is defined for regrtest 
(issue #12250):

"The implicit timeout in regrtest.py makes it harder to write automated
test scripts for 3rd party modules. First, you have to remember to
set --timeout=0 for long running tests. Then, you have to remember
not to use the --timeout option when compiling --without-threads.

I'd much prefer that there's no timeout unless explicitly specified.
For the buildbots, I think this could be done in the Makefile."

First I replaced the hardcoded constant in regrtest.py by a command line 
argument (--timeout=3600) in the TESTOPTS variable of the Makefile, so if you 
call regrtest directly (without make), there is no more default timeout.

But today I saw a a buildbot timeout without any traceback: a possible hang in 
test_io on "x86 FreeBSD 7.2 3.x" buildbot, "command timed out: 3900 seconds 
without output". I realized that some buildbots (all buildbots?) override the 
TESTOPTS variable ("make buildbottest TESTOPTS= TESTPYTHONOPTS=" for "x86 
FreeBSD 7.2 3.x").

I moved the timeout option from TESTOPTS to a new variable: TESTTIMEOUT, with 
a default value of 1 hour (3600 seconds). The timeout is now only used for 
"make buildbottest". Use TESTTIMEOUT=0 to disable completly regrtest timeout. 
The timeout argument if ignored (with a warning) if 
faulthandler.dump_tracebacks_later() function is missing (e.g. if Python is 
compiled without threads).

Please, don't disable the TESTTIMEOUT option for your buildbot. You may want 
to use a shorter or longer timeout, it just have to be smaller than the 
buildbot timeout (3900 seconds by default, 1 hour 6 minutes, just a little bit 
more than regrtest timeout: 1 hour). If you want a timeout longer than 1 hour, 
you can to change TESTTIMEOUT and the buildbot timeout (I don't know how to 
configure the buildbot timeout).

I didn't touch test, testall, testuniversal, quicktest make rules, which don't 
use the regrtest timeout anymore. I would prefer to use the same timeout 
options for all test rules in TESTOPTS, but it doesn't work because some 
buildbots do override TESTOPTS variable.

Victor

From steve at pearwood.info  Mon Jun  6 13:37:53 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 06 Jun 2011 21:37:53 +1000
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <20110606121340.0f7a3464@pitrou.net>
References: <20110521170725.51eab5f9@pitrou.net>
	<4DE2643D.3070906@v.loewis.de>	<20110604205026.374c0019@pitrou.net>
	<4DEB2306.5080206@v.loewis.de>	<20110605123313.64a4a720@pitrou.net>	<19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>	<1307347881.3506.1.camel@localhost.localdomain>	<4DEC90CA.8070604@v.loewis.de>
	<20110606121340.0f7a3464@pitrou.net>
Message-ID: <4DECBC11.8060407@pearwood.info>

Antoine Pitrou wrote:
> On Mon, 06 Jun 2011 10:33:14 +0200
> "Martin v. L?wis" <martin at v.loewis.de> wrote:
[...]
>> How can you make such claims when several people have indicated that
>> the howto *actually* helped them?
> 
> The point here is that the examples in that document are very poor
> (the only substantial example actually duplicates existing
> functionality - in a sub-optimal manner - without even mentioning the
> existence of said functionality), and the technical explanations are
> nearly non-existent. So I'll happy stand by my claims.

You know, for the amount of discussion about whether or not the doc is 
worth keeping, we probably could have fixed all the problems with it :)

I believe that "status quo wins" is worth applying here. In the absence 
of evidence that the HOWTO is actively harmful, we should keep it.

I'm of two minds whether it should go into the wiki. I would hate for 
the wiki to become the place where bad docs go to die, but on the other 
hand putting it in the wiki may encourage lightweight incremental fixes.

I think the Socket HOWTO is important enough to fix, not throw out. I 
also dislike link-rot, and throwing it out causes link-rot. I'd rather 
see a bunch of concrete bug reports for the HOWTO than just a dismissive 
"throw it out and start again".


> The Python documentation isn't meant to host any potentially helpful
> document, however flawed. We have the Internet for that.

I think it is unfair to dismiss the document as "potentially" helpful 
when a number of people have said that it *actually* did help them.



-- 
Steven

From p.f.moore at gmail.com  Mon Jun  6 13:48:52 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 6 Jun 2011 12:48:52 +0100
Subject: [Python-Dev] Buildbots and regrtest timeout
In-Reply-To: <201106061229.46577.victor.stinner@haypocalc.com>
References: <201106061229.46577.victor.stinner@haypocalc.com>
Message-ID: <BANLkTikHhTn9tJJuOfHKt0KGdZknthf5bA@mail.gmail.com>

On 6 June 2011 11:29, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> Stephan Krah asked me to change how the default timeout is defined for regrtest
> (issue #12250):
>
> "The implicit timeout in regrtest.py makes it harder to write automated
> test scripts for 3rd party modules. First, you have to remember to
> set --timeout=0 for long running tests. Then, you have to remember
> not to use the --timeout option when compiling --without-threads.
>
> I'd much prefer that there's no timeout unless explicitly specified.
> For the buildbots, I think this could be done in the Makefile."
>
> First I replaced the hardcoded constant in regrtest.py by a command line
> argument (--timeout=3600) in the TESTOPTS variable of the Makefile, so if you
> call regrtest directly (without make), there is no more default timeout.
>
> But today I saw a a buildbot timeout without any traceback: a possible hang in
> test_io on "x86 FreeBSD 7.2 3.x" buildbot, "command timed out: 3900 seconds
> without output". I realized that some buildbots (all buildbots?) override the
> TESTOPTS variable ("make buildbottest TESTOPTS= TESTPYTHONOPTS=" for "x86
> FreeBSD 7.2 3.x").
>
> I moved the timeout option from TESTOPTS to a new variable: TESTTIMEOUT, with
> a default value of 1 hour (3600 seconds). The timeout is now only used for
> "make buildbottest". Use TESTTIMEOUT=0 to disable completly regrtest timeout.
> The timeout argument if ignored (with a warning) if
> faulthandler.dump_tracebacks_later() function is missing (e.g. if Python is
> compiled without threads).
>
> Please, don't disable the TESTTIMEOUT option for your buildbot. You may want
> to use a shorter or longer timeout, it just have to be smaller than the
> buildbot timeout (3900 seconds by default, 1 hour 6 minutes, just a little bit
> more than regrtest timeout: 1 hour). If you want a timeout longer than 1 hour,
> you can to change TESTTIMEOUT and the buildbot timeout (I don't know how to
> configure the buildbot timeout).
>
> I didn't touch test, testall, testuniversal, quicktest make rules, which don't
> use the regrtest timeout anymore. I would prefer to use the same timeout
> options for all test rules in TESTOPTS, but it doesn't work because some
> buildbots do override TESTOPTS variable.

How does this impact Windows buildbots? As they don't use the
makefile, did you add an override to the Windows scripts as well, or
will WIndows now use the default of no timeout?

Paul.

From stephen at xemacs.org  Mon Jun  6 15:59:04 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 6 Jun 2011 22:59:04 +0900
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <1307347881.3506.1.camel@localhost.localdomain>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<20110605123313.64a4a720@pitrou.net>
	<19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>
	<1307347881.3506.1.camel@localhost.localdomain>
Message-ID: <19948.56616.844528.700142@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > So did you read the discussion before posting?

Yes.  It's rude to assume that those who disagree with you are
irresponsible and uninformed.  Would you please stop it?

 > The sockets HOWTO *doesn't* serve both groups.
 > Actually, I would argue that it serves neither of them.

I know that is your opinion, because I've read your posts.  I
disagree.  The document is imperfect, but for me it served a certain
purpose.  Therefore I stand with the camp that says improving the
document is the way to go.


From solipsis at pitrou.net  Mon Jun  6 16:06:12 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 06 Jun 2011 16:06:12 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <19948.56616.844528.700142@uwakimon.sk.tsukuba.ac.jp>
References: <20110521170725.51eab5f9@pitrou.net>
	<4DE2643D.3070906@v.loewis.de> <20110604205026.374c0019@pitrou.net>
	<4DEB2306.5080206@v.loewis.de> <20110605123313.64a4a720@pitrou.net>
	<19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>
	<1307347881.3506.1.camel@localhost.localdomain>
	<19948.56616.844528.700142@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1307369172.3506.2.camel@localhost.localdomain>

Le lundi 06 juin 2011 ? 22:59 +0900, Stephen J. Turnbull a ?crit :
> Antoine Pitrou writes:
> 
>  > So did you read the discussion before posting?
> 
> Yes.  It's rude to assume that those who disagree with you are
> irresponsible and uninformed.  Would you please stop it?
> 
>  > The sockets HOWTO *doesn't* serve both groups.
>  > Actually, I would argue that it serves neither of them.
> 
> I know that is your opinion, because I've read your posts.  I
> disagree.  The document is imperfect, but for me it served a certain
> purpose.  Therefore I stand with the camp that says improving the
> document is the way to go.

Don't hesitate to contribute.

Regards

Antoine.



From merwok at netwok.org  Mon Jun  6 18:01:37 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 06 Jun 2011 18:01:37 +0200
Subject: [Python-Dev] packaging landed in stdlib
In-Reply-To: <BANLkTikg_OswAKYE-+r6iLyoA2-yzikBfQ@mail.gmail.com>
References: <BANLkTikg_OswAKYE-+r6iLyoA2-yzikBfQ@mail.gmail.com>
Message-ID: <4DECF9E1.6080303@netwok.org>

Hi,

My GSoC student made me realize that it could be quite difficult to work
with the packaging module codebase when you?re familiar with the
distutils code.  I wrote a blog post about file-level changes from
distutils to packaging, to help people find their way:
https://wokslog.wordpress.com/2011/06/04/distutils-diff/

Regards

From kevinjcoyne at hotmail.com  Mon Jun  6 19:43:07 2011
From: kevinjcoyne at hotmail.com (kevin coyne)
Date: Mon, 6 Jun 2011 13:43:07 -0400
Subject: [Python-Dev] Help requested for Python ISO Standard
Message-ID: <BLU135-W16102361A7A9D705018946A4600@phx.gbl>


To whom it may concern:
I am working on an ISO Annex of Vulnerabilities for the Python language and am asking for help getting a list of language features that exhibit:
unspecified behavior, undefined behavior, or implementation defined behavior. I am also searching for a list of deprecated features. Guido (copied) had suggested I contact you folks for help as we are coming up on an ISO deadline and I have not been successful finding this information.
Thanks very much in advance for your help!
Regards,

Kevin Coyne
Cell: 703.901.6774


 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110606/e6738037/attachment.html>

From db3l.net at gmail.com  Mon Jun  6 20:55:28 2011
From: db3l.net at gmail.com (David Bolen)
Date: Mon, 06 Jun 2011 14:55:28 -0400
Subject: [Python-Dev] Buildbots and regrtest timeout
References: <201106061229.46577.victor.stinner@haypocalc.com>
Message-ID: <m24o42x00v.fsf@valheru.db3l.homeip.net>

Victor Stinner <victor.stinner at haypocalc.com> writes:

> But today I saw a a buildbot timeout without any traceback: a possible hang in 
> test_io on "x86 FreeBSD 7.2 3.x" buildbot, "command timed out: 3900 seconds 
> without output". I realized that some buildbots (all buildbots?) override the 
> TESTOPTS variable ("make buildbottest TESTOPTS= TESTPYTHONOPTS=" for "x86 
> FreeBSD 7.2 3.x").
(...)
> Please, don't disable the TESTTIMEOUT option for your buildbot. You may want 
> to use a shorter or longer timeout, (...)

Just a note, given the phrasing above.  None of this is anything that
I for example, as a buildbot operator, am actively controlling.  That
command, including the environment variable overrides, is exactly as
provided by the master for a given test run.  So I'd think you could
adjust it if needed through changes in the master build configuration
and probably without having to add an environment variable or 

Per Paul's follow-up on Windows, buildbot under Windows seems to
impose a 1200s idle timeout (just for no output), but I'm not positive
how it's calculated.  The test process itself has never, I'm pretty
sure, specified a timeout to regrtest (via the test.bat, rt.bat,
regrtest.py path).  (Oh, I guess the --timeout option itself in
regrtest is fairly new)

So if there's a change in defaults for regrtest that will change
Windows behavior implicitly, and I believe at this point the buildbots
will be inconsistent, since you're only overriding the regrtest
default in a subset of buildbot types.

-- David


From barry at python.org  Mon Jun  6 21:23:14 2011
From: barry at python.org (Barry Warsaw)
Date: Mon, 6 Jun 2011 15:23:14 -0400
Subject: [Python-Dev] FWD: gpg keys have problems
In-Reply-To: <20110605152025.GA25276@panix.com>
References: <20110605152025.GA25276@panix.com>
Message-ID: <20110606152314.3130cd6f@neurotica.wooz.org>

On Jun 05, 2011, at 08:20 AM, Aahz wrote:

>> >From your python download page you need to update the public keys to not
>> use the faulty MD5 digest algorithm.  (see the link listed below)
>> 
>> $ gpg --import pubkeys.txt
>> gpg: key 6A45C816: public key "Anthony Baxter <anthony at interlink.com.au>" imported
>> gpg: WARNING: digest algorithm MD5 is deprecated
>> gpg: please see http://www.gnupg.org/faq/weak-digest-algos.html for more information
>> gpg: key ED9D77D5: public key "Barry A. Warsaw <barry at warsaw.us>" imported
>> gpg: Total number processed: 2
>> gpg:               imported: 2  (RSA: 1)
>> gpg: no ultimately trusted keys found

This only looks like a problem with Anthony's key.  He could update his key,
but OTOH probably has little incentive to just for Python release management.
Anthony was release manager for 2.5, but Martin took that over, and also,
Python 2.5 is very near EOL even for security releases.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110606/b9e5178e/attachment.pgp>

From skip at pobox.com  Mon Jun  6 22:36:11 2011
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 6 Jun 2011 15:36:11 -0500
Subject: [Python-Dev] Help requested for Python ISO Standard
In-Reply-To: <BLU135-W16102361A7A9D705018946A4600@phx.gbl>
References: <BLU135-W16102361A7A9D705018946A4600@phx.gbl>
Message-ID: <19949.14907.982155.4509@montanaro.dyndns.org>


    kevin> I am working on an ISO Annex of Vulnerabilities for the Python
    kevin> language and am asking for help getting a list of language
    kevin> features that exhibit:

    kevin> unspecified behavior, undefined behavior, or implementation
    kevin> defined behavior.  I am also searching for a list of deprecated
    kevin> features.
 
One place to search:

    http://svn.python.org/projects/python/branches/py3k/Lib/test/crashers/

-- 
Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/

From anurag.chourasia at gmail.com  Mon Jun  6 23:23:25 2011
From: anurag.chourasia at gmail.com (Anurag Chourasia)
Date: Mon, 6 Jun 2011 17:23:25 -0400
Subject: [Python-Dev] AIX 5.3 - Build of Python 2.7.1
Message-ID: <BANLkTi=o8GqsdO47XqG5nG20745g_1uMvA@mail.gmail.com>

Hi All,

Python2.7.1 build on AIX 5.3 ML 12 is failing at the make step with the
error indicated below

de -I./Include -I /opt/freeware/include -I /opt/freeware/include/readline -I
/opt/freeware/include/ncurses  -DPy_BUILD_CORE -o Parser/tokenizer_pgen.o
Parser/tokenizer_pgen.c
        gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3
-Wall -Wstrict-prototypes  -I. -IInclude -I./Include -I
/opt/freeware/include -I /opt/freeware/include/readline -I
/opt/freeware/include/ncurses  -DPy_BUILD_CORE -o Parser/printgrammar.o
Parser/printgrammar.c
        gcc -pthread -c -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3
-Wall -Wstrict-prototypes  -I. -IInclude -I./Include -I
/opt/freeware/include -I /opt/freeware/include/readline -I
/opt/freeware/include/ncurses  -DPy_BUILD_CORE -o Parser/pgenmain.o
Parser/pgenmain.c
        gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
Parser/acceler.o  Parser/grammar1.o  Parser/listnode.o  Parser/node.o
 Parser/parser.o  Parser/parsetok.o  Parser/bitset.o  Parser/metagrammar.o
 Parser/firstsets.o  Parser/grammar.o  Parser/pgen.o Objects/obmalloc.o
 Python/mysnprintf.o  Python/pyctype.o  Parser/tokenizer_pgen.o
 Parser/printgrammar.o  Parser/pgenmain.o -ldl  -o Parser/pgen
ld: 0711-593 SEVERE ERROR: Symbol C_BSTAT (entry 559) in object
Parser/grammar1.o:
        The symbol refers to a csect with symbol number 0, which was not
        found. The new symbol cannot be associated with a csect and
        is being ignored.
ld: 0711-593 SEVERE ERROR: Symbol C_BSTAT (entry 579) in object
Parser/firstsets.o:
        The symbol refers to a csect with symbol number 0, which was not
        found. The new symbol cannot be associated with a csect and
        is being ignored.
collect2: ld returned 12 exit status
make: 1254-004 The error code from the last command is 1.

Is there anything I could try to get past this?

Here is the configuration and make step that i utilized

./configure ?enable-shared --disable-ipv6 --with-gcc=gcc CPPFLAGS="-I
/opt/freeware/include -I /opt/freeware/include/readline -I
/opt/freeware/include/ncurses"


make


Regards,
Anurag
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110606/2bb7cdc4/attachment-0001.html>

From mr_mcl at live.co.uk  Tue Jun  7 00:15:51 2011
From: mr_mcl at live.co.uk (C McL)
Date: Mon, 6 Jun 2011 23:15:51 +0100
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <19948.56616.844528.700142@uwakimon.sk.tsukuba.ac.jp>
References: <20110521170725.51eab5f9@pitrou.net>
	<4DE2643D.3070906@v.loewis.de>,<20110604205026.374c0019@pitrou.net>
	<4DEB2306.5080206@v.loewis.de>, <20110605123313.64a4a720@pitrou.net>,
	<19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>,
	<1307347881.3506.1.camel@localhost.localdomain>,
	<19948.56616.844528.700142@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <SNT131-w215B9EC71894ED9B5593F1DC600@phx.gbl>



> Date: Mon, 6 Jun 2011 22:59:04 +0900
> From: stephen at xemacs.org
> To: solipsis at pitrou.net
> CC: python-dev at python.org
> Subject: Re: [Python-Dev] The socket HOWTO
> 
> I know that is your opinion, because I've read your posts.  I
> disagree.  The document is imperfect, but for me it served a certain
> purpose.  Therefore I stand with the camp that says improving the
> document is the way to go.

+1. I've been reading the postings on this discussion intently, as I have had experience with the socket HOWTO when I was first learning about sockets. I agree with the view that Martin v. L?wis expressed, that as a beginner I didn't read too much into the text at first because I was more concerned with trying out the examples and getting used to writing the code and such. I would also say that, I wasn't too bothered if the guide never went into too much detail about all the terms it was mentioning, it isn't after all a definitive guide on sockets, but the terms can always be googled later if one so wished.

I wholeheartedly disagree with removing it, that would be a real shame and I dislike the idea of moving it to the wiki (I cannot even remember ever visiting the wiki). I may not be a Python Guru but I think my "n00bishness" helps in this particular discussion and I would say I would have to agree to an improvement over the suggested alternatives.

Craig

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110606/fd3906a0/attachment.html>

From kevinjcoyne at hotmail.com  Tue Jun  7 00:14:49 2011
From: kevinjcoyne at hotmail.com (kevin coyne)
Date: Mon, 6 Jun 2011 18:14:49 -0400
Subject: [Python-Dev] Help requested for Python ISO Standard
In-Reply-To: <19949.14907.982155.4509@montanaro.dyndns.org>
References: <BLU135-W16102361A7A9D705018946A4600@phx.gbl>,
	<19949.14907.982155.4509@montanaro.dyndns.org>
Message-ID: <BLU135-W18284D601429E1AC8BAC24A4600@phx.gbl>


Skip:
Thanks, appreciate the link  I've checked them all out and some may be useful to my task.
Regards, 

Kevin Coyne
Cell: 703.901.6774




> Date: Mon, 6 Jun 2011 15:36:11 -0500
> To: kevinjcoyne at hotmail.com
> CC: python-dev at python.org; l.wagone at radium.ncsc.mil; guido at python.org
> Subject: Re: [Python-Dev] Help requested for Python ISO Standard
> From: skip at pobox.com
> 
> 
>     kevin> I am working on an ISO Annex of Vulnerabilities for the Python
>     kevin> language and am asking for help getting a list of language
>     kevin> features that exhibit:
> 
>     kevin> unspecified behavior, undefined behavior, or implementation
>     kevin> defined behavior.  I am also searching for a list of deprecated
>     kevin> features.
>  
> One place to search:
> 
>     http://svn.python.org/projects/python/branches/py3k/Lib/test/crashers/
> 
> -- 
> Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110606/8e27e488/attachment.html>

From ncoghlan at gmail.com  Tue Jun  7 03:50:22 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Jun 2011 11:50:22 +1000
Subject: [Python-Dev] Help requested for Python ISO Standard
In-Reply-To: <BLU135-W18284D601429E1AC8BAC24A4600@phx.gbl>
References: <BLU135-W16102361A7A9D705018946A4600@phx.gbl>
	<19949.14907.982155.4509@montanaro.dyndns.org>
	<BLU135-W18284D601429E1AC8BAC24A4600@phx.gbl>
Message-ID: <BANLkTimd1n-1qEnmGQSS7xpTKay4RjW=kg@mail.gmail.com>

On Tue, Jun 7, 2011 at 8:14 AM, kevin coyne <kevinjcoyne at hotmail.com> wrote:
> Skip:
> Thanks, appreciate the link ?I've checked them all out and some may be
> useful to my task.

Another two scans to try would be to look for "cpython" in the test
suite and "impl-detail" in the documentation.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Tue Jun  7 04:47:37 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Jun 2011 12:47:37 +1000
Subject: [Python-Dev] [Python-checkins] cpython: Issue #12040: Expose a
 new attribute `sentinel` on instances of
In-Reply-To: <E1QTdig-0005FF-KR@dinsdale.python.org>
References: <E1QTdig-0005FF-KR@dinsdale.python.org>
Message-ID: <BANLkTikA8VgvuNhFDhMq--oVKQTe5oWZ_w@mail.gmail.com>

On Tue, Jun 7, 2011 at 3:36 AM, antoine.pitrou
<python-checkins at python.org> wrote:
> + ? ?@property
> + ? ?def sentinel(self):
> + ? ? ? ?'''
> + ? ? ? ?Return a file descriptor (Unix) or handle (Windows) suitable for
> + ? ? ? ?waiting for process termination.
> + ? ? ? ?'''
> + ? ? ? ?try:
> + ? ? ? ? ? ?return self._sentinel
> + ? ? ? ?except AttributeError:
> + ? ? ? ? ? ?raise ValueError("process not started")
> +

We should probably follow this model for threading.Thread.ident as
well (i.e. throwing an exception rather than returning None if the
thread hasn't been started yet).

Also, for runtime state errors, we tend to use RuntimeError rather
than ValueError (e.g. see the errors thrown by
contextlib._GeneratorContextManager)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From g.brandl at gmx.net  Tue Jun  7 08:57:10 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 07 Jun 2011 08:57:10 +0200
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <E1QTmqU-0004iT-VX@dinsdale.python.org>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
Message-ID: <iski44$q0m$1@dough.gmane.org>

On 06/07/11 05:20, brett.cannon wrote:
> http://hg.python.org/cpython/rev/fc282e375703
> changeset:   70695:fc282e375703
> user:        Brett Cannon <brett at python.org>
> date:        Mon Jun 06 20:20:36 2011 -0700
> summary:
>   Remove some extraneous parentheses and swap the comparison order to
> prevent accidental assignment.
> 
> Silences a warning from LLVM/clang 2.9.

Swapping the comparison order here seems a bit inconsistent to me. There are
lots of others around (e.g. "len == 0" in the patch context below). Why is
this one so special?

I think that another developer even got told off once for these kinds of
comparisons.

I hope the Clang warning is only about the parentheses.

Georg

> files:
>   Modules/arraymodule.c |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c
> --- a/Modules/arraymodule.c
> +++ b/Modules/arraymodule.c
> @@ -2091,7 +2091,7 @@
>      if (len == 0) {
>          return PyUnicode_FromFormat("array('%c')", (int)typecode);
>      }
> -    if ((typecode == 'u'))
> +    if ('u' == typecode)
>          v = array_tounicode(a, NULL);
>      else
>          v = array_tolist(a, NULL);




From solipsis at pitrou.net  Tue Jun  7 10:25:01 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 7 Jun 2011 10:25:01 +0200
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org>
Message-ID: <20110607102501.6f112546@pitrou.net>

On Tue, 07 Jun 2011 08:57:10 +0200
Georg Brandl <g.brandl at gmx.net> wrote:
> On 06/07/11 05:20, brett.cannon wrote:
> > http://hg.python.org/cpython/rev/fc282e375703
> > changeset:   70695:fc282e375703
> > user:        Brett Cannon <brett at python.org>
> > date:        Mon Jun 06 20:20:36 2011 -0700
> > summary:
> >   Remove some extraneous parentheses and swap the comparison order to
> > prevent accidental assignment.
> > 
> > Silences a warning from LLVM/clang 2.9.
> 
> Swapping the comparison order here seems a bit inconsistent to me. There are
> lots of others around (e.g. "len == 0" in the patch context below). Why is
> this one so special?

Agreed. Either we do it wholesale (I find these "reversed" comparisons
a bit ugly myself) or there's no point doing it on a single occurrence.

Regards

Antoine.



From lukasz at langa.pl  Tue Jun  7 10:47:45 2011
From: lukasz at langa.pl (=?iso-8859-2?Q?=A3ukasz_Langa?=)
Date: Tue, 7 Jun 2011 10:47:45 +0200
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <SNT131-w215B9EC71894ED9B5593F1DC600@phx.gbl>
References: <20110521170725.51eab5f9@pitrou.net>
	<4DE2643D.3070906@v.loewis.de>,
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>,
	<20110605123313.64a4a720@pitrou.net>,
	<19948.35147.349129.663487@uwakimon.sk.tsukuba.ac.jp>,
	<1307347881.3506.1.camel@localhost.localdomain>,
	<19948.56616.844528.700142@uwakimon.sk.tsukuba.ac.jp>
	<SNT131-w215B9EC71894ED9B5593F1DC600@phx.gbl>
Message-ID: <AFA8731B-3551-44C5-9EBA-5F6D37B70C84@langa.pl>


Wiadomo?? napisana przez C McL w dniu 2011-06-07, o godz. 00:15:

> I cannot even remember ever visiting the wiki.

FWIW neither can I. The Wiki link on the front page is below Jobs and Merchandise so it's easy to miss it altogether ;-)

-- 
Best regards,
?ukasz Langa
Senior Systems Architecture Engineer

IT Infrastructure Department
Grupa Allegro Sp. z o.o.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110607/fcd79093/attachment.html>

From mal at egenix.com  Tue Jun  7 11:03:22 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 07 Jun 2011 11:03:22 +0200
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <iski44$q0m$1@dough.gmane.org>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org>
Message-ID: <4DEDE95A.2010603@egenix.com>

Georg Brandl wrote:
> On 06/07/11 05:20, brett.cannon wrote:
>> http://hg.python.org/cpython/rev/fc282e375703
>> changeset:   70695:fc282e375703
>> user:        Brett Cannon <brett at python.org>
>> date:        Mon Jun 06 20:20:36 2011 -0700
>> summary:
>>   Remove some extraneous parentheses and swap the comparison order to
>> prevent accidental assignment.
>>
>> Silences a warning from LLVM/clang 2.9.
> 
> Swapping the comparison order here seems a bit inconsistent to me. There are
> lots of others around (e.g. "len == 0" in the patch context below). Why is
> this one so special?
>
> I think that another developer even got told off once for these kinds of
> comparisons.
>
> I hope the Clang warning is only about the parentheses.

I agree with Georg: "if ('u' == typecode)" is not well readable,
since you usually put the variable part on the left and the constant
part on the right of an equal comparison.

If clang warns about this, clang needs to be fixed, not our
C code :-)

> Georg
> 
>> files:
>>   Modules/arraymodule.c |  2 +-
>>   1 files changed, 1 insertions(+), 1 deletions(-)
>>
>>
>> diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c
>> --- a/Modules/arraymodule.c
>> +++ b/Modules/arraymodule.c
>> @@ -2091,7 +2091,7 @@
>>      if (len == 0) {
>>          return PyUnicode_FromFormat("array('%c')", (int)typecode);
>>      }
>> -    if ((typecode == 'u'))
>> +    if ('u' == typecode)
>>          v = array_tounicode(a, NULL);
>>      else
>>          v = array_tolist(a, NULL);
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 07 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2011-05-23: Released eGenix mx Base 3.2.0      http://python.egenix.com/
2011-05-25: Released mxODBC 3.1.1              http://python.egenix.com/
2011-06-20: EuroPython 2011, Florence, Italy               13 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From eliben at gmail.com  Tue Jun  7 11:47:13 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Tue, 7 Jun 2011 12:47:13 +0300
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <4DEDE95A.2010603@egenix.com>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org> <4DEDE95A.2010603@egenix.com>
Message-ID: <BANLkTim0Mnp1=wFV-_rTOkZiTPqpHMgDzw@mail.gmail.com>

>> Swapping the comparison order here seems a bit inconsistent to me. There are
>> lots of others around (e.g. "len == 0" in the patch context below). Why is
>> this one so special?
>>
>> I think that another developer even got told off once for these kinds of
>> comparisons.
>>
>> I hope the Clang warning is only about the parentheses.
>
> I agree with Georg: "if ('u' == typecode)" is not well readable,
> since you usually put the variable part on the left and the constant
> part on the right of an equal comparison.
>
> If clang warns about this, clang needs to be fixed, not our
> C code :-)
>

+1

Placing the constant first in a comparison is a fundamental style
issue. Personally I also don't like doing that, but whatever way is
chosen must be consistent. It's definitely wrong to change this in a
single place. We have PEP-7 for these things!

AFAIK, Clang doesn't produce a warning for this, at least without
special static-analysis warning levels.

Eli

From ncoghlan at gmail.com  Tue Jun  7 12:10:10 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Jun 2011 20:10:10 +1000
Subject: [Python-Dev] [Python-checkins] cpython: Issue #12040: Expose a
 new attribute `sentinel` on instances of
In-Reply-To: <20110607102256.1dabc92d@pitrou.net>
References: <E1QTdig-0005FF-KR@dinsdale.python.org>
	<BANLkTikA8VgvuNhFDhMq--oVKQTe5oWZ_w@mail.gmail.com>
	<20110607102256.1dabc92d@pitrou.net>
Message-ID: <BANLkTinkg2J3f-wYE6J1Kadx7c-Z0yc4VQ@mail.gmail.com>

On Tue, Jun 7, 2011 at 6:22 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 7 Jun 2011 12:47:37 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> We should probably follow this model for threading.Thread.ident as
>> well (i.e. throwing an exception rather than returning None if the
>> thread hasn't been started yet).
>
> Well, Process.ident returns None as well ;)

Hmm, I guess in that case my preference for keeping the Thread and
Process APIs reasonably consistent just edges out my dislike of
returning None when a "you shouldn't do that" exception would be more
appropriate.

>> Also, for runtime state errors, we tend to use RuntimeError rather
>> than ValueError (e.g. see the errors thrown by
>> contextlib._GeneratorContextManager)
>
> Well, it depends. For example, closed files raise a ValueError when you
> try to do operations on them. I tend to think of RuntimeError as
> something that happens without any clear responsibility from the user,
> although that's debatable (recursion errors can occur because of a
> programming bug, but also simply because a structure is nested much too
> deeply).

Yeah, I only thought of the closed file counterexample after I had
already posted. I guess I just have a slight personal preference for
RuntimeError to flag state problems as they're less likely to be
caught up in an overly broad try statement in user code (as catching
RuntimeError seems to be less common than catching ValueError).

> Of course, current coding practice in other multiprocessing.Process
> methods and properties doesn't help, since it uses asserts to guard
> against misuse!

I have a vague recollection of an existing tracker issue complaining
about that... or am I just remembering the one that addressed similar
problems in threading?

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From glyph at twistedmatrix.com  Tue Jun  7 05:54:12 2011
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Mon, 6 Jun 2011 20:54:12 -0700
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <4DEC04CA.1060608@v.loewis.de>
References: <20110521170725.51eab5f9@pitrou.net>
	<4DE2643D.3070906@v.loewis.de>	<20110604205026.374c0019@pitrou.net>
	<4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
	<4DEC04CA.1060608@v.loewis.de>
Message-ID: <1E59F13B-852E-4DB7-8FB2-D7B067C90920@twistedmatrix.com>

On Jun 5, 2011, at 3:35 PM, Martin v. L?wis wrote:

> And that's all fine. I still claim that you have to *understand*
> sockets in order to use it properly. By this, I mean stuff like
> "what is a TCP connection? how is it established?", "how is UDP
> different from TCP?", "when data arrives, what layers of software
> does it go through?", "what is a port number?", etc.

Yes, these are all excellent concepts to be familiar with.  But the word "socket" (and the socket HOWTO) refers to a specific way to interface with those concepts, the Berkeley socket API: <http://en.wikipedia.org/wiki/Berkeley_sockets>.  Which you don't have to know anything about if you're going to use Twisted.  You should know about IPC in general, and TCP/UDP specifically if you're going to use Twisted, but sockets are completely optional.

Also, I feel that I should point out that the sockets HOWTO does not cover even a single one of these concepts in any useful depth.  If you think that these are what it should be explaining, it needs some heavy editing.  Here's what it has to say about each one:

> what is a TCP connection?

The only place that the characters "TCP" appear in the entire document is in the phrase "... which is completely different from TCP_NODELAY ...".  Nowhere is a TCP connection explained at a conceptual level, except to say that it's something a web browser does.

> how is UDP different from TCP?

The phrase "UDP" never appears in the HOWTO.  DGRAM sockets get a brief mention as "anything else" in the sentence: "... you?ll get better behavior and performance from a STREAM socket than anything else ...".  (To be fair, I do endorse teaching that "the difference between TCP and UDP is that you should not use UDP" to anyone not sufficiently advanced to read the relevant reference documentation themselves.)

> when data arrives, what layers of software does it go through?

There's no discussion of this that I can find at all.

> what is a port number?

Aside from a few comments in the code examples, the only discussion of port numbers is "low number ports are usually reserved for ?well known? services (HTTP, SNMP etc)."

It would be very good to have a "Python networking overview" somewhere that explained this stuff at a very high level, and described how data might get into or out of your program, with links to things like the socket HOWTO that describe more specific techniques.  This would be useful because most commonly, I think that data will get into Python network programs via WSGI, not direct sockets or anything like Twisted.

To be clear, having read it now: I do _not_ agree with Antoine that this document should be deleted.  I dimly recall that it helped me understand some things in the very early days of Twisted.  While it's far from perfect, it might help someone in a similar situation understand those things as well today.  I just found it interesting that the main concepts one would associate with such a HOWTO are nowhere to be found :).

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110606/ecf3cee0/attachment.html>

From eliben at gmail.com  Tue Jun  7 19:37:44 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Tue, 7 Jun 2011 20:37:44 +0300
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <1E59F13B-852E-4DB7-8FB2-D7B067C90920@twistedmatrix.com>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
	<4DEC04CA.1060608@v.loewis.de>
	<1E59F13B-852E-4DB7-8FB2-D7B067C90920@twistedmatrix.com>
Message-ID: <BANLkTikdhh_rGk-Erh+NjUEMkSp5Th=NEQ@mail.gmail.com>

<snip>
> Yes, these are all excellent concepts to be familiar with. ?But the word
> "socket" (and the socket HOWTO) refers to a specific way to interface with
> those concepts, the Berkeley socket API:
> <http://en.wikipedia.org/wiki/Berkeley_sockets>. ?Which you don't have to
> know anything about if you're going to use Twisted. ?You should know about
> IPC in general, and TCP/UDP specifically if you're going to use Twisted, but
> sockets are completely optional.
> Also, I feel that I should point out that the sockets HOWTO does not cover
> even a single one of these concepts in any useful depth. ?If you think that
> these are what it should be explaining, it needs some heavy editing. ?Here's
> what it has to say about each one:
>
> what is a TCP connection?
>
> The only place that the characters "TCP" appear in the entire document is in
> the phrase "...?which is completely different from?TCP_NODELAY ...".
> ?Nowhere is a TCP connection explained at a conceptual level, except to say
> that it's something a web browser does.
>
> how is UDP?different from TCP?
>
> The phrase "UDP" never appears in the HOWTO. ?DGRAM sockets get a brief
> mention as "anything else" in the sentence: "... you?ll get better behavior
> and performance from a STREAM socket than anything else ...". ?(To be fair,
> I do endorse teaching that "the difference between TCP and UDP is that you
> should not use UDP" to anyone not sufficiently advanced to read the relevant
> reference documentation themselves.)
>
> when data arrives, what layers of software?does it go through?
>
> There's no discussion of this that I can find at all.
>
> what is a port number?
>
> Aside from a few comments in the code examples, the only discussion of port
> numbers is "low number ports are usually reserved for ?well known? services
> (HTTP,?SNMP etc)."
> It would be very good to have a "Python networking overview" somewhere that
> explained this stuff at a very high level, and described how data might get
> into or out of your program, with links to things like the socket HOWTO that
> describe more specific techniques. ?This would be useful because most
<snip>

Just be careful not to reproduce http://www.apress.com/9781590593714 :-)
These things tend to get out of hand very quickly.

Eli

From amauryfa at gmail.com  Tue Jun  7 21:43:57 2011
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 7 Jun 2011 21:43:57 +0200
Subject: [Python-Dev] AIX 5.3 - Build of Python 2.7.1
In-Reply-To: <BANLkTi=o8GqsdO47XqG5nG20745g_1uMvA@mail.gmail.com>
References: <BANLkTi=o8GqsdO47XqG5nG20745g_1uMvA@mail.gmail.com>
Message-ID: <BANLkTi=R-nGtVF85j_QXH1cPHQhJJwrcSg@mail.gmail.com>

Hi,

2011/6/6 Anurag Chourasia <anurag.chourasia at gmail.com>:
> ld: 0711-593 SEVERE ERROR: Symbol C_BSTAT (entry 559) in object
> Parser/grammar1.o:

A quick Google search reveals a probable issue with gcc on recent AIX systems.
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=46072

A lot of users report this issue on various programs. Nothing Python
can do about.
You could try to disable the -g option.

-- 
Amaury Forgeot d'Arc

From guido at python.org  Tue Jun  7 23:11:41 2011
From: guido at python.org (Guido van Rossum)
Date: Tue, 7 Jun 2011 14:11:41 -0700
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <BANLkTim0Mnp1=wFV-_rTOkZiTPqpHMgDzw@mail.gmail.com>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org> <4DEDE95A.2010603@egenix.com>
	<BANLkTim0Mnp1=wFV-_rTOkZiTPqpHMgDzw@mail.gmail.com>
Message-ID: <BANLkTimXvy=df8h3sOryz0LN4Ad+HYf3u2gB_bfk7ZqWUwiqzA@mail.gmail.com>

On Tue, Jun 7, 2011 at 2:47 AM, Eli Bendersky <eliben at gmail.com> wrote:
>>> Swapping the comparison order here seems a bit inconsistent to me. There are
>>> lots of others around (e.g. "len == 0" in the patch context below). Why is
>>> this one so special?
>>>
>>> I think that another developer even got told off once for these kinds of
>>> comparisons.
>>>
>>> I hope the Clang warning is only about the parentheses.
>>
>> I agree with Georg: "if ('u' == typecode)" is not well readable,
>> since you usually put the variable part on the left and the constant
>> part on the right of an equal comparison.
>>
>> If clang warns about this, clang needs to be fixed, not our
>> C code :-)
>>
>
> +1
>
> Placing the constant first in a comparison is a fundamental style
> issue. Personally I also don't like doing that, but whatever way is
> chosen must be consistent. It's definitely wrong to change this in a
> single place. We have PEP-7 for these things!

Right. I personally really despise putting the constant first.

> AFAIK, Clang doesn't produce a warning for this, at least without
> special static-analysis warning levels.

CLang shouldn't force our hand here.

-- 
--Guido van Rossum (python.org/~guido)

From alexander.belopolsky at gmail.com  Tue Jun  7 23:42:17 2011
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 7 Jun 2011 17:42:17 -0400
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <BANLkTimXvy=df8h3sOryz0LN4Ad+HYf3u2gB_bfk7ZqWUwiqzA@mail.gmail.com>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org> <4DEDE95A.2010603@egenix.com>
	<BANLkTim0Mnp1=wFV-_rTOkZiTPqpHMgDzw@mail.gmail.com>
	<BANLkTimXvy=df8h3sOryz0LN4Ad+HYf3u2gB_bfk7ZqWUwiqzA@mail.gmail.com>
Message-ID: <BANLkTikM4kQDzptEx=Pbw_xtiVRLQ6jQYA@mail.gmail.com>

..
>>> I agree with Georg: "if ('u' == typecode)" is not well readable,
>>> since you usually put the variable part on the left and the constant
>>> part on the right of an equal comparison.

I appear to be in the minority here, but this particular example does
not strike me as egregiously unreadable.  To the contrary, by bringing
the constant to the front, this form saves me from having to read to
the end of the line.  The same mental economy appears when constants
are brought to the front of chained if-elif cases in Python:

if 'a' == typecode:
  ..
elif 'b' == typecode:
  ..

is slightly more readable than the more traditional alternative.
Probably because I can mentally ignore the "== typecode" part and see
the switch structure more clearly.

Either way, I don't see a big issue here and I would keep "len == 0"
intact even if I reordered typecode == 'u' as Brett did.  The only
consistency that I would enforce is to use the same order in the
chained if-elif cases, but otherwise this should be left to the
discretion of the author.

From victor.stinner at haypocalc.com  Wed Jun  8 00:23:20 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 8 Jun 2011 00:23:20 +0200
Subject: [Python-Dev] Byte filenames in the posix module on Windows
Message-ID: <201106080023.20795.victor.stinner@haypocalc.com>

Hi,

Last november, we "decided" (right?) to deprecate bytes filenames in the posix 
module on Windows in Python 3.2 and drop the support in 3.3: see "Removal of 
Win32 ANSI API" thread on python-dev. Python 3.2 has been released, so we  
should shift the versions numbers.

I would like to take care of this. I propose three steps:

1) Remove the bytes implementation of each function (when the code is not 
shared with other OSes), decode bytes from MBCS and reuse the Unicode code.

2) Deprecate the bytes code in Python 3.3

3) Drop bytes support in Python 3.4

(I'm only talking about the posix module on Windows)

The first step should not change anything for the user, but it will remove a 
lot of duplicated code. I expect something like removing the half of the code 
specific to Windows in the posix module.

If we decide to keep bytes filenames on Windows, we can stop before the second 
step.

--

One important point is the choice of the error handler: I would like to mimic 
the ANSI API and so I will use MultiByteToWideChar() with flags=0 (e.g. MBCS 
codec with ignore error handler, but see also the issue #12281 !). The MBCS 
codec uses the ANSI code page which can be a multibyte encoding, like ShiftJIS 
(cp932 with a japanese locale).

os.fsdecode(), PyUnicode_DecodeFSDefault() and PyUnicode_FSDecoder() use the 
strict error handler to decode filenames on Windows. We may also use strict in 
the posix module. I'm +0 for this because it warns the developer (and user?) 
that he/she is doing something really bad.

--

I would like to simplify posixmodule.c because I saw that it is difficult to 
patch it to fix bugs (you have to patch two functions, or more, for each fix). A 
recent example is the os.stat() symlink issue on Windows:
http://bugs.python.org/issue12084

Since Windows 2000, filenames are stored as Unicode internally (e.g. VFAT and 
NTFS use UTF-16), the ANSI API was kept for backward compatibility (for lazy 
developers!). If you use bytes for filenames on Windows, you may get encode 
errors because the ANSI code page is a small subset of Unicode.

For your information, I have a last -huge- pending patch to only use Unicode 
in the import machinery, issue #11619 ;-) Using this patch, you can use 
characters not encodable to the ANSI code page in your module name/path, yeah! 
But I am not completly convinced that we need this patch...

Victor

From dmalcolm at redhat.com  Tue Jun  7 23:35:29 2011
From: dmalcolm at redhat.com (David Malcolm)
Date: Tue, 07 Jun 2011 17:35:29 -0400
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <4DEDE95A.2010603@egenix.com>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org>  <4DEDE95A.2010603@egenix.com>
Message-ID: <1307482529.16924.331.camel@surprise>

On Tue, 2011-06-07 at 11:03 +0200, M.-A. Lemburg wrote:
> Georg Brandl wrote:
> > On 06/07/11 05:20, brett.cannon wrote:
> >> http://hg.python.org/cpython/rev/fc282e375703
> >> changeset:   70695:fc282e375703
> >> user:        Brett Cannon <brett at python.org>
> >> date:        Mon Jun 06 20:20:36 2011 -0700
> >> summary:
> >>   Remove some extraneous parentheses and swap the comparison order to
> >> prevent accidental assignment.
> >>
> >> Silences a warning from LLVM/clang 2.9.
> > 
> > Swapping the comparison order here seems a bit inconsistent to me. There are
> > lots of others around (e.g. "len == 0" in the patch context below). Why is
> > this one so special?
> >
> > I think that another developer even got told off once for these kinds of
> > comparisons.
> >
> > I hope the Clang warning is only about the parentheses.
> 
> I agree with Georg: "if ('u' == typecode)" is not well readable,
> since you usually put the variable part on the left and the constant
> part on the right of an equal comparison.

[FWIW, I'm one of the reprobates that likes to put the constant on the
LHS when I'm coding in C, but I see I'm in the minority here]

I know that this style is unpopular, but if it helps, try mentally
pronouncing  "=="  in C as  "is the value of".

In this example, when I read that line, my mind is thinking:
 
   "if 'u' is the value of typecode"

After ~12 years of doing this, it comes naturally.  I appreciate that
this may come across as weird though :)

[snip]

Hope this is helpful
Dave


From rdmurray at bitdance.com  Wed Jun  8 03:16:32 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 07 Jun 2011 21:16:32 -0400
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
	swap the comparison order to
In-Reply-To: <BANLkTikM4kQDzptEx=Pbw_xtiVRLQ6jQYA@mail.gmail.com>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org> <4DEDE95A.2010603@egenix.com>
	<BANLkTim0Mnp1=wFV-_rTOkZiTPqpHMgDzw@mail.gmail.com>
	<BANLkTimXvy=df8h3sOryz0LN4Ad+HYf3u2gB_bfk7ZqWUwiqzA@mail.gmail.com>
	<BANLkTikM4kQDzptEx=Pbw_xtiVRLQ6jQYA@mail.gmail.com>
Message-ID: <20110608011632.DB76E25012E@webabinitio.net>

On Tue, 07 Jun 2011 17:42:17 -0400, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
> ..
> >>> I agree with Georg: "if ('u' == typecode)" is not well readable,
> >>> since you usually put the variable part on the left and the constant
> >>> part on the right of an equal comparison.
> 
> I appear to be in the minority here, but this particular example does
> not strike me as egregiously unreadable.  To the contrary, by bringing
> the constant to the front, this form saves me from having to read to
> the end of the line.  The same mental economy appears when constants
> are brought to the front of chained if-elif cases in Python:
> 
> if 'a' == typecode:
>   ..
> elif 'b' == typecode:
>   ..
> 
> is slightly more readable than the more traditional alternative.
> Probably because I can mentally ignore the "== typecode" part and see
> the switch structure more clearly.

I don't do much C coding, so I don't have the right to an opinion on
that (FWIW, I find constant-first jarring).  But I'd hate to see the
above in python code.  The fact that you like it because it makes it
easier to read as a switch-like statement should instead tell you that
it is time to refactor the code.

--
R. David Murray           http://www.bitdance.com

From tjreedy at udel.edu  Wed Jun  8 04:43:32 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 07 Jun 2011 22:43:32 -0400
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <1307482529.16924.331.camel@surprise>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>	<iski44$q0m$1@dough.gmane.org>
	<4DEDE95A.2010603@egenix.com> <1307482529.16924.331.camel@surprise>
Message-ID: <ismnkj$1et$1@dough.gmane.org>

On 6/7/2011 5:35 PM, David Malcolm wrote:

> I know that this style is unpopular, but if it helps, try mentally
> pronouncing  "=="  in C as  "is the value of".
>
> In this example, when I read that line, my mind is thinking:
>
>     "if 'u' is the value of typecode"
>
> After ~12 years of doing this, it comes naturally.  I appreciate that
> this may come across as weird though :)

Whereas I read it as 'has the value' (or just 'is' ;=).

Not being tempted to reverse in Python is one advantage of not having 
assignment expressions.

-- 
Terry Jan Reedy


From brett at python.org  Wed Jun  8 05:07:49 2011
From: brett at python.org (Brett Cannon)
Date: Tue, 7 Jun 2011 20:07:49 -0700
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <iski44$q0m$1@dough.gmane.org>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org>
Message-ID: <BANLkTimN-TT+U=sTYjDM9Q5E789bROma7w@mail.gmail.com>

On Mon, Jun 6, 2011 at 23:57, Georg Brandl <g.brandl at gmx.net> wrote:

> On 06/07/11 05:20, brett.cannon wrote:
> > http://hg.python.org/cpython/rev/fc282e375703
> > changeset:   70695:fc282e375703
> > user:        Brett Cannon <brett at python.org>
> > date:        Mon Jun 06 20:20:36 2011 -0700
> > summary:
> >   Remove some extraneous parentheses and swap the comparison order to
> > prevent accidental assignment.
> >
> > Silences a warning from LLVM/clang 2.9.
>
> Swapping the comparison order here seems a bit inconsistent to me. There
> are
> lots of others around (e.g. "len == 0" in the patch context below). Why is
> this one so special?
>
>
Old habit on how to do comparisons in C. Because C treats assignment as an
expression it means comparisons can accidentally become an assignment if you
accidentally leave out an = sign. By reversing this order it is simply not
possible to have that silent bug and instead you would get a compiler error
about trying to assign to a constant.

I'll revert that part of the change.


> I think that another developer even got told off once for these kinds of
> comparisons.
>
> I hope the Clang warning is only about the parentheses.
>

Yes, Clang only warned about the parentheses.

-Brett


>
> Georg
>
> > files:
> >   Modules/arraymodule.c |  2 +-
> >   1 files changed, 1 insertions(+), 1 deletions(-)
> >
> >
> > diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c
> > --- a/Modules/arraymodule.c
> > +++ b/Modules/arraymodule.c
> > @@ -2091,7 +2091,7 @@
> >      if (len == 0) {
> >          return PyUnicode_FromFormat("array('%c')", (int)typecode);
> >      }
> > -    if ((typecode == 'u'))
> > +    if ('u' == typecode)
> >          v = array_tounicode(a, NULL);
> >      else
> >          v = array_tolist(a, NULL);
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110607/475d2f66/attachment.html>

From steve at pearwood.info  Wed Jun  8 13:04:48 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 08 Jun 2011 21:04:48 +1000
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <ismnkj$1et$1@dough.gmane.org>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>	<iski44$q0m$1@dough.gmane.org>	<4DEDE95A.2010603@egenix.com>
	<1307482529.16924.331.camel@surprise>
	<ismnkj$1et$1@dough.gmane.org>
Message-ID: <4DEF5750.3000005@pearwood.info>

Terry Reedy wrote:
> On 6/7/2011 5:35 PM, David Malcolm wrote:
> 
>> I know that this style is unpopular, but if it helps, try mentally
>> pronouncing  "=="  in C as  "is the value of".
>>
>> In this example, when I read that line, my mind is thinking:
>>
>>     "if 'u' is the value of typecode"
>>
>> After ~12 years of doing this, it comes naturally.  I appreciate that
>> this may come across as weird though :)
> 
> Whereas I read it as 'has the value' (or just 'is' ;=).


Am I the only one who reads == as "equals"?



-- 
Steven

From dsdale24 at gmail.com  Wed Jun  8 17:01:22 2011
From: dsdale24 at gmail.com (Darren Dale)
Date: Wed, 8 Jun 2011 11:01:22 -0400
Subject: [Python-Dev] Can we improve support for abstract base classes with
	desciptors
Message-ID: <BANLkTi=A_VuqQpNr+2VowEXLRENC7p-z2Q@mail.gmail.com>

I would like to try to address some shortfalls with the way python deals with
abstract base classes containing descriptors. I originally was just concerned
with improving support for defining abstract properties with the decorator
syntax and converting between abstract and concrete properties, but recently
realized that the problem extends to descriptors in general.

ABCs
----

First, a bit of background may be in order. An abstract base class is defined
by specifying its metaclass as ABCMeta (or a subclass thereof)::

    class MyABC(metaclass=ABCMeta):
        @abstractmethod
	def foo(self):
	    pass

When trying to instantiate MyABC or any of its subclasses, ABCMeta inspects the
current class namespace for items tagged with __isabstractmethod__=True::

    class ABCMeta(type):
    #[...]
        def __new__(mcls, name, bases, namespace):
            cls = super().__new__(mcls, name, bases, namespace)
            # Compute set of abstract method names
            abstracts = {name
                         for name, value in namespace.items()
                         if getattr(value, "__isabstractmethod__", False)}

ABCMeta then checks if any of the base classes define any items tagged with
__isabstractmethod__ and whether they remain abstract in the current
class namespace::

            for base in bases:
                for name in getattr(base, "__abstractmethods__", set()):
                    value = getattr(cls, name, None)
                    if getattr(value, "__isabstractmethod__", False):
                        abstracts.add(name)
            cls.__abstractmethods__ = frozenset(abstracts)

In Objects/typeobject.c, __abstractmethods__ is actually a descriptor, and
setting it gives the type a chance to set an internal flag specifying if it
has any abstract methods defined. When object_new is called in typeobject.c,
the flag is checked and an error is raised if any abstract methods were
identified.

Issues with ABCs and descriptors
--------------------------------

In order for this scheme to work, ABCMeta needs to identify all of the abstract
methods, but there are some limitations when we consider descriptors. For
example, Python's property is a composite object, whose behavior is defined by
the getter, setter, and deleter methods with which it is composed. Since there
is already an @abstractmethod decorator, I would have suspected that defining
abstract properties would be intuitive::

    class MyABC(metaclass=ABCMeta):
        @abstractmethod
	def _get_foo(self):
	    pass
	@abstractmethod
	def _set_foo(self, val):
	    pass
	foo = property(_get_foo, _set_foo)

        @property
	@abstractmethod
	def bar(self):
	    pass
	@bar.setter
	@abstractmethod
	def bar(self, val):
	    pass

Ideally, one would want the flexibility of defining a concrete getter and an
abstract setter, for example. However, ABCMeta does not inspect the descriptors
of a class to see if they contain any abstract methods. It only inspects the
descriptor itself for a True __isabstractmethod__ attribute. This places the
burdon on every descriptor implementation to provide its own support for ABC
compatibility. For example, support for abstract properties was attempted by
adding abstractproperty to the abc module. abstractproperty subclasses the
property builtin (as opposed to the relationship between every other abstract
and concrete class in the python language). Here is the definition of
abstractproperty, in its entirety (modulo docstrings)::

    class abstractproperty(property):
        __isabstractmethod__ = True

A number of problems manifest with this approach, and I think they all can be
traced to the fact that the abstractedness of a descriptor is currently not
dependent upon the abstractedness of the methods with which it is
composed. The documentation for abstractproperty doesn't suggest using
@abstractmethod::

        class C(metaclass=ABCMeta):
            def getx(self): ...
            def setx(self, value): ...
            x = abstractproperty(getx, setx)

which leads to Issue #1: What is abstract about C.x? How does a subclass of C
know whether it needs to override the getter or setter?

Issue #2: The decorator syntax cannot be used to convert an abstract property
into a concrete one. (This relates to Issue #1: how would a descriptor even know
when such a conversion would be appropriate?) Running the following code::

    from abc import ABCMeta, abstractmethod, abstractproperty

    class AbstractFoo(metaclass=ABCMeta):
        @abstractproperty
        def bar(self):
            return 1
        @bar.setter
        def bar(self, val):
            pass

    class ConcreteFoo(AbstractFoo):
        @AbstractFoo.bar.getter
        def bar(self):
            return 1
        @bar.setter
        def bar(self, val):
            pass
    foo = ConcreteFoo()

yields::

    TypeError: Can't instantiate abstract class ConcreteFoo with abstract
    methods bar

Issue #3: The following class *is* instantiable, even though
AbstractFoo declared that a setter for bar is required::

    class ConcreteFoo(AbstractFoo):
        @property
	def bar(self):
	    pass

Previous attempt to improve abc.abstractproperty
------------------------------------------------

It seems to me that the strategy used by abc.abstractproperty is fundamentally
ill-advised. I explored the possibility of extending abstractproperty,
redefining its getter, setter, and deleter methods such that they would work in
conjunction with the @abstractmethod decorator and yield an instance of the
builtin property once all abstract methods were replaced with concrete ones
(http://bugs.python.org/issue11610). Issues #1 and #2 were addressed, but there
were still problems with that approach. It did not address Issue #3, and it
also introduced a new issue, #4::

    class AbstractFoo(metaclass=ABCMeta):
        @abstractproperty
	# bar would be an abstractproperty, even though the getter is concrete
        def bar(self):
            return 1
        @bar.setter
	# bar.setter inspected the getter and the new setter, did not identify
	# any abstract methods, and thus returned an instance of the built-in
	# property
        def bar(self, val):
            pass
	@bar.deleter
	# bar is a concrete property, its deleter decorator does not know it
	# is supposed to check for abstract methods, so it will return an
	# instance of the built-in property:
	@abstractmethod
	def bar(self):
	    pass

By the time the deleter was specified, bar was a concrete property, which does
not know it should return an instance of abstractproperty (in part because the
inheritance diagram for property/abstractproperty is inverted). Thus,
AbstractFoo was instantiable, even though it shouldn't be.

Finally, issue #5: the current approach taken by ABCMeta and abstractproperty
places the burdon on descriptors to identify themselves to ABCMeta as
abstract. Considering the issues encountered with abstractproperty, this may be
an onerous requirement.

There has been a fair amount of discussion at http://bugs.python.org/issue11610
, which can be summarized as a) concerns about maintaining backward
compatibility, and b) objections to requiring @abstractmethod to specify that a
method being passed to abstractproperty is abstract.

Extending ABCMeta: A Promising Way Forward
------------------------------------------

I think the key is to focus on Issue #3. ABCMeta needs to be improved to
recognize descriptor objects, both in the current namespace as well as the base
classes, and to identify any abstract methods associated with the
descriptors. I suggest the following approach in ABCMeta::

    def __new__(mcls, name, bases, namespace):
        cls = super().__new__(mcls, name, bases, namespace)
        # Compute set of abstract method names

        def isdescriptor(val):
            return hasattr(val, '__get__') or hasattr(val, '__set__') \
                    or hasattr(val, '__delete__')
        def getabstracts(ns):
            return [name
                    for name, value in ns.items()
                    if getattr(value, "__isabstractmethod__", False)]

        abstracts = getabstracts(namespace)
        for item, val in namespace.items():
            # further inspect descriptors for abstract methods:
            if isdescriptor(val):
                ## unfortunately, can't import inspect:
                #from inspect import getmembers
                #d = dict(getmembers(val))
		## so instead, use the following:
                d = dict((k, getattr(val, k, None)) for k in dir(val))
                for name in getabstracts(d):
		    # add the abstract descriptor methods to the list:
                    abstracts.append('%s.%s'%(item, name))
        for base in bases:
            for name in getattr(base, "__abstractmethods__", set()):
                if '.' in name:
                    # base class identified a descriptor abstract method:
                    k, v = name.split('.')
                    desc = getattr(cls, k, None)
                    val = getattr(desc, v, None)
                else:
                    val = getattr(cls, name, None)
                if val is None or getattr(val, "__isabstractmethod__", False):
                    abstracts.append(name)
        cls.__abstractmethods__ = frozenset(abstracts)

I rolled everything into __new__ just to keep it as simple as possible for the
sake of discussion. Python already provides the rest of the framework needed
for descriptors to work properly with ABCs. This implementation actually works;
I've tested it with an existing python-3.2 install::

    from abc import ABCMeta, abstractmethod

    class AbstractFoo(metaclass=ABCMeta):
        @property
        @abstractmethod
        def bar(self):
            return 1
        @bar.setter
        @abstractmethod
        def bar(self, val):
            pass

    >>> abstractfoo = AbstractFoo()
    Traceback (most recent call last):
      File "temp.py", line 17, in <module>
        abstractfoo = AbstractFoo()
    TypeError: Can't instantiate abstract class AbstractFoo with abstract
    methods bar.fget, bar.fset

as expected. Note the more informative error message indicating what about the
bar property is abstract. Also::

    class ConcreteFoo(AbstractFoo):
        @AbstractFoo.bar.getter
        def bar(self):
            return 1

    >>> foo = ConcreteFoo()
    Traceback (most recent call last):
      File "temp.py", line 24, in <module>
        foo = ConcreteFoo()
    TypeError: Can't instantiate abstract class ConcreteFoo with abstract
    methods bar.fset

So issue #1 is addressed, since we are explicitly specifying which descriptor
methods are abstract. Issue #2 has been addressed, since the following class is
instantiable::

    class ConcreteFoo(AbstractFoo):
        @AbstractFoo.bar.getter
        def bar(self):
            return 1
	@bar.setter
	def bar(self, val):
	    pass

Issue #3 is also addressed. In the following example, even though I redefine
the bar property as a readonly property, ABCMeta recognizes that a setter is
needed::

    class ConcreteFoo(AbstractFoo):
        @property
        def bar(self):
            return 1

    >>> foo = ConcreteFoo()
    Traceback (most recent call last):
      File "temp.py", line 24, in <module>
        foo = ConcreteFoo()
    TypeError: Can't instantiate abstract class ConcreteFoo with abstract
    methods bar.fset

Issue #4 (introduced in a previous attempt to solve the problem
using abstractproperty) is also addressed::

    class AbstractFoo(metaclass=ABCMeta):
        @property
        def bar(self):
            return 1
        @bar.setter
        def bar(self, val):
            pass
        @bar.deleter
        @abstractmethod
        def bar(self):
            pass

    >>> abstractfoo = AbstractFoo()
    Traceback (most recent call last):
      File "temp.py", line 15, in <module>
        abstractfoo = AbstractFoo()
    TypeError: Can't instantiate abstract class AbstractFoo with abstract
    methods bar.fdel

Finally, this approach addresses Issue #5 by holding ABCMeta responsible for
identifying the abstractedness of descriptor methods, rather than placing that
burdon on the desciptor objects to identify themselves as abstract. If ABCMeta
were extended as above to identify abstract methods associated with
descriptors, third parties would simply decorate methods used to compose the
descriptors with @abstractmethod.

This change to ABCMeta would not effect the behavior of abstractproperty, so
backward compatibility would be maintained in that respect. But I think
abstractproperty should be deprecated, or at the very least removed from the
documentation. The documentation for @abstractmethod in >=python-3.3 should be
extended to provide examples with properties/descriptors. The syntax would be
backward compatible with older python versions, but with <python-3.3 ABCMeta
would simply not recognize descriptors' abstract methods. This leads to one
source of potential forward compatibility::

    class AbstractFoo(metaclass=ABCMeta):
        @property
	@abstractmethod
        def bar(self):
            return 1

    class ConcreteFoo(AbstractFoo):
        pass

Both above classes would be instantiable with <python-3.3, but not with
>=python3.3. In my opinion, this is a feature: python-3.3 has identified a bug
in ConcreteFoo. The developer would not have tagged that method as abstract
unless they had intended for consumers of AbstractFoo to provide a concrete
implementation.


Darren

From ncoghlan at gmail.com  Wed Jun  8 17:01:48 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Jun 2011 01:01:48 +1000
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <BANLkTikdhh_rGk-Erh+NjUEMkSp5Th=NEQ@mail.gmail.com>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
	<4DEC04CA.1060608@v.loewis.de>
	<1E59F13B-852E-4DB7-8FB2-D7B067C90920@twistedmatrix.com>
	<BANLkTikdhh_rGk-Erh+NjUEMkSp5Th=NEQ@mail.gmail.com>
Message-ID: <BANLkTi=ir=8uhgDdy5r9-wUXrpKCEheqBw@mail.gmail.com>

On Wed, Jun 8, 2011 at 3:37 AM, Eli Bendersky <eliben at gmail.com> wrote:
> Just be careful not to reproduce http://www.apress.com/9781590593714 :-)
> These things tend to get out of hand very quickly.

At the level Glyph and Martin are talking about, you're more likely to
end up with http://authors.phptr.com/tanenbaumcn4/ :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Wed Jun  8 17:06:55 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Jun 2011 01:06:55 +1000
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <1E59F13B-852E-4DB7-8FB2-D7B067C90920@twistedmatrix.com>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
	<4DEC04CA.1060608@v.loewis.de>
	<1E59F13B-852E-4DB7-8FB2-D7B067C90920@twistedmatrix.com>
Message-ID: <BANLkTi=UxzyyRN+tDL8dYcQVkZfpB+_HBQ@mail.gmail.com>

On Tue, Jun 7, 2011 at 1:54 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> how is UDP?different from TCP?
>
> The phrase "UDP" never appears in the HOWTO. ?DGRAM sockets get a brief
> mention as "anything else" in the sentence: "... you?ll get better behavior
> and performance from a STREAM socket than anything else ...". ?(To be fair,
> I do endorse teaching that "the difference between TCP and UDP is that you
> should not use UDP" to anyone not sufficiently advanced to read the relevant
> reference documentation themselves.)

And if UDP starts sounding tempting due to excessively high latency,
these days it's worth looking up the specs for the interplanetary
internet instead.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Wed Jun  8 17:12:18 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Jun 2011 01:12:18 +1000
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <1307482529.16924.331.camel@surprise>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org> <4DEDE95A.2010603@egenix.com>
	<1307482529.16924.331.camel@surprise>
Message-ID: <BANLkTimrqQ3VfCkpBa8HvPeB+igTOLB06w@mail.gmail.com>

On Wed, Jun 8, 2011 at 7:35 AM, David Malcolm <dmalcolm at redhat.com> wrote:
> After ~12 years of doing this, it comes naturally. ?I appreciate that
> this may come across as weird though :)

I actually thought Brett's rationale in the checkin comment was
reasonable (if you get in the habit of putting constants on the left,
then the classic "'=' instead of '=='" typo is a compiler error
instead of a reassignment).

Call it a +0 in favour of letting people put constants on the left in
C code if they prefer it that way, so long as any given if/elif chain
is consistent in the style it uses.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From alexander.belopolsky at gmail.com  Wed Jun  8 17:14:04 2011
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 8 Jun 2011 11:14:04 -0400
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <4DEF5750.3000005@pearwood.info>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org> <4DEDE95A.2010603@egenix.com>
	<1307482529.16924.331.camel@surprise>
	<ismnkj$1et$1@dough.gmane.org> <4DEF5750.3000005@pearwood.info>
Message-ID: <BANLkTiniS1p5ehVb0KnQLz0nCBLN+37sBg@mail.gmail.com>

On Wed, Jun 8, 2011 at 7:04 AM, Steven D'Aprano <steve at pearwood.info> wrote:
..
>> Whereas I read it as 'has the value' (or just 'is' ;=).
>
> Am I the only one who reads == as "equals"?

If you are, you are the only one who reads it correctly.  Consider

>>> a = 2
>>> a == 2.0
True

From rdmurray at bitdance.com  Wed Jun  8 17:19:02 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 08 Jun 2011 11:19:02 -0400
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
	swap the comparison order to
In-Reply-To: <4DEF5750.3000005@pearwood.info>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org> <4DEDE95A.2010603@egenix.com>
	<1307482529.16924.331.camel@surprise>
	<ismnkj$1et$1@dough.gmane.org> <4DEF5750.3000005@pearwood.info>
Message-ID: <20110608151903.7F4D425012E@webabinitio.net>

On Wed, 08 Jun 2011 21:04:48 +1000, Steven D'Aprano <steve at pearwood.info> wrote:
> Terry Reedy wrote:
> > On 6/7/2011 5:35 PM, David Malcolm wrote:
> > 
> >> I know that this style is unpopular, but if it helps, try mentally
> >> pronouncing  "=="  in C as  "is the value of".
> >>
> >> In this example, when I read that line, my mind is thinking:
> >>
> >>     "if 'u' is the value of typecode"
> >>
> >> After ~12 years of doing this, it comes naturally.  I appreciate that
> >> this may come across as weird though :)
> > 
> > Whereas I read it as 'has the value' (or just 'is' ;=).
> 
> 
> Am I the only one who reads == as "equals"?

No :)

Especially considering that Python actually has an 'is' operator....

--
R. David Murray           http://www.bitdance.com

From ncoghlan at gmail.com  Wed Jun  8 17:55:28 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Jun 2011 01:55:28 +1000
Subject: [Python-Dev] Can we improve support for abstract base classes
 with desciptors
In-Reply-To: <BANLkTi=A_VuqQpNr+2VowEXLRENC7p-z2Q@mail.gmail.com>
References: <BANLkTi=A_VuqQpNr+2VowEXLRENC7p-z2Q@mail.gmail.com>
Message-ID: <BANLkTimPwrsRqzaAnknc-y9Qe1=ke3Gp9A@mail.gmail.com>

On Thu, Jun 9, 2011 at 1:01 AM, Darren Dale <dsdale24 at gmail.com> wrote:
[snip excellent analysis of the problem]

I have some suggestions regarding a few details of your current code,
but your basic proposal looks sound to me.

I would tweak __new__ along the following lines though:

   def __new__(mcls, name, bases, namespace):
       cls = super().__new__(mcls, name, bases, namespace)
       # Compute set of abstract method names

       # CHANGE 1: refactor descriptor and abstract method scan to
happen in a single pass
       def is_descriptor(value):
           return (hasattr(value, '__get__') or hasattr(value, '__set__')
                     or hasattr(value, '__delete__'))
       def is_abstract(value):
           return getattr(value, "__isabstractmethod__", False)
       def get_abstract_names_for_item(item):
           name, value = item
           if is_abstract(value):
               return [name]
           elif is_descriptor(value):
               # CHANGE 2: Deliberately ignore descriptors on the
descriptor objects
               # CHANGE 3: Use new-style string formatting
               return ['{}.{}'.format(name, attr) for attr in
dir(value) if is_abstract(getattr(value, attr))]
           return []
       def get_abstract_names(ns):
           names = []
           for item in ns.items():
               names.extend(get_abstract_names_for_item(item))
           return names

       abstract_names = get_abstract_names(namespace.items())

       for base in bases:
           for name in getattr(base, "__abstractmethods__", ()):
               # CHANGE 4: Using rpartition better tolerates weird
naming in the metaclass
               # (weird naming in descriptors will still blow up in
the earlier search for abstract names)
               descr_name, is_descr, attr = name.rpartition('.')
               if is_descr:
                   # base class identified a descriptor abstract method:
                   descr = getattr(cls, descr_name, None)
                   val = getattr(descr, attr, None)
               else:
                   val = getattr(cls, name, None)
               if val is None or is_abstract(val):
                   abstract_names.append(name)

       cls.__abstractmethods__ = frozenset(abstract_names)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From debatem1 at gmail.com  Wed Jun  8 20:07:15 2011
From: debatem1 at gmail.com (geremy condra)
Date: Wed, 8 Jun 2011 11:07:15 -0700
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <BANLkTikdhh_rGk-Erh+NjUEMkSp5Th=NEQ@mail.gmail.com>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
	<4DEC04CA.1060608@v.loewis.de>
	<1E59F13B-852E-4DB7-8FB2-D7B067C90920@twistedmatrix.com>
	<BANLkTikdhh_rGk-Erh+NjUEMkSp5Th=NEQ@mail.gmail.com>
Message-ID: <BANLkTimVNHVZJPmi4Ex99QOwx6tWF8ETzA@mail.gmail.com>

On Tue, Jun 7, 2011 at 10:37 AM, Eli Bendersky <eliben at gmail.com> wrote:
>
> Just be careful not to reproduce http://www.apress.com/9781590593714 :-)
> These things tend to get out of hand very quickly.

You say that like it's a bad thing. The first few chapters of that
would make a great replacement for the howto.

Geremy Condra

From tom.whittock at gmail.com  Wed Jun  8 21:30:36 2011
From: tom.whittock at gmail.com (Tom Whittock)
Date: Wed, 8 Jun 2011 20:30:36 +0100
Subject: [Python-Dev] fatal error callback issue
Message-ID: <BANLkTimRQdpr+2SkTZy9npCzwtKWqPAr8g@mail.gmail.com>

I'm writing in regards to http://bugs.python.org/issue1195571

I'm embedding Python in my application and ran into a need for this
functionality. I wrote a similar patch myself, and was about to submit
it. When I searched for similar issues I found that this one has been
available since 2005.

I'd really like to help get this patch approved and integrated into
the python sources. I'm sure many other python embedders have run into
this in the past - it's a legitimate concern for any process which is
not 100% hosted in the Python world.

Thanks.
Tom.

From dsdale24 at gmail.com  Thu Jun  9 00:51:52 2011
From: dsdale24 at gmail.com (Darren Dale)
Date: Wed, 8 Jun 2011 18:51:52 -0400
Subject: [Python-Dev] Can we improve support for abstract base classes
 with desciptors
In-Reply-To: <BANLkTimPwrsRqzaAnknc-y9Qe1=ke3Gp9A@mail.gmail.com>
References: <BANLkTi=A_VuqQpNr+2VowEXLRENC7p-z2Q@mail.gmail.com>
	<BANLkTimPwrsRqzaAnknc-y9Qe1=ke3Gp9A@mail.gmail.com>
Message-ID: <BANLkTinJLsE0whSoP4iQWhLrjS1GBMpdYg@mail.gmail.com>

On Wed, Jun 8, 2011 at 11:55 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Thu, Jun 9, 2011 at 1:01 AM, Darren Dale <dsdale24 at gmail.com> wrote:
> [snip excellent analysis of the problem]
>
> I have some suggestions regarding a few details of your current code,
> but your basic proposal looks sound to me.
>
> I would tweak __new__ along the following lines though:
[snip]

Thank you, I agree. Concerning the following block:

> ? ? ? def get_abstract_names(ns):
> ? ? ? ? ? names = []
> ? ? ? ? ? for item in ns.items():
> ? ? ? ? ? ? ? names.extend(get_abstract_names_for_item(item))
> ? ? ? ? ? return names
>
> ? ? ? abstract_names = get_abstract_names(namespace.items())

That should be "get_abstract_names(namespace)", since ns.items() gets
called again in the for loop. I think the get_abstract_names function
isn't needed though, since it is only ever called that one time. Any
reason not replace the above block with::

        abstract_names = []
        for item in namespace.items():
            abstract_names.extend(get_abstract_names_for_item(item))

> ? ? ? for base in bases:
> ? ? ? ? ? for name in getattr(base, "__abstractmethods__", ()):
> ? ? ? ? ? ? ? # CHANGE 4: Using rpartition better tolerates weird
> naming in the metaclass
> ? ? ? ? ? ? ? # (weird naming in descriptors will still blow up in
> the earlier search for abstract names)

Could you provide an example of weird naming?

Darren

From victor.stinner at haypocalc.com  Thu Jun  9 02:46:19 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 09 Jun 2011 02:46:19 +0200
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Merge
In-Reply-To: <E1QUT8v-0007LV-9s@dinsdale.python.org>
References: <E1QUT8v-0007LV-9s@dinsdale.python.org>
Message-ID: <1307580379.14102.2.camel@marge>

Le jeudi 09 juin 2011 ? 02:30 +0200, brian.curtin a ?crit :
> http://hg.python.org/cpython/rev/f1509fc75435
> changeset:   70715:f1509fc75435
> branch:      2.7
> parent:      70661:6e7a98cfcfab
> user:        Brian Curtin <brian at python.org>
> date:        Wed Jun 08 19:29:53 2011 -0500
> summary:
>   Merge

FYI this commit has only one parent, it's not a merge. It should have a
changelog describing the patch.

... whereas the following commit has two parents, it's a merge:

> http://hg.python.org/cpython/rev/567f30527913
> changeset:   70714:567f30527913
> parent:      70712:964d0d65a2a9
> parent:      70713:88e318166eaf
> user:        Brian Curtin <brian at python.org>
> date:        Wed Jun 08 18:43:57 2011 -0500
> summary:
>   Fix #11583. Changed os.path.isdir to use GetFileAttributes (...)

You can use "merge", "Merge 3.2", or other similir changelog. I prefer
to use "(Merge 3.2)Fix #11583. Changed os.path._isdir ..." : copy/paste
the changelog of the original commit with "(Merge 3.2) " prefix.

Victor


From tjreedy at udel.edu  Thu Jun  9 02:47:17 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 08 Jun 2011 20:47:17 -0400
Subject: [Python-Dev] fatal error callback issue
In-Reply-To: <BANLkTimRQdpr+2SkTZy9npCzwtKWqPAr8g@mail.gmail.com>
References: <BANLkTimRQdpr+2SkTZy9npCzwtKWqPAr8g@mail.gmail.com>
Message-ID: <isp56n$d28$1@dough.gmane.org>

On 6/8/2011 3:30 PM, Tom Whittock wrote:
> I'm writing in regards to http://bugs.python.org/issue1195571
>
> I'm embedding Python in my application and ran into a need for this
> functionality. I wrote a similar patch myself, and was about to submit
> it. When I searched for similar issues I found that this one has been
> available since 2005.
>
> I'd really like to help get this patch approved and integrated into
> the python sources. I'm sure many other python embedders have run into
> this in the past - it's a legitimate concern for any process which is
> not 100% hosted in the Python world.

Add a comment like this to the issue itself. Also review the most recent 
patch, considering Victor's comments. Is it better than yours? Can you 
improve it? Can you test it?

-- 
Terry Jan Reedy


From ncoghlan at gmail.com  Thu Jun  9 04:01:01 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Jun 2011 12:01:01 +1000
Subject: [Python-Dev] Can we improve support for abstract base classes
 with desciptors
In-Reply-To: <BANLkTinJLsE0whSoP4iQWhLrjS1GBMpdYg@mail.gmail.com>
References: <BANLkTi=A_VuqQpNr+2VowEXLRENC7p-z2Q@mail.gmail.com>
	<BANLkTimPwrsRqzaAnknc-y9Qe1=ke3Gp9A@mail.gmail.com>
	<BANLkTinJLsE0whSoP4iQWhLrjS1GBMpdYg@mail.gmail.com>
Message-ID: <BANLkTi=+0z8N70d7xnVGo+5=BCCOQphjHA@mail.gmail.com>

On Thu, Jun 9, 2011 at 8:51 AM, Darren Dale <dsdale24 at gmail.com> wrote:
> That should be "get_abstract_names(namespace)", since ns.items() gets
> called again in the for loop. I think the get_abstract_names function
> isn't needed though, since it is only ever called that one time. Any
> reason not replace the above block with::
>
> ? ? ? ?abstract_names = []
> ? ? ? ?for item in namespace.items():
> ? ? ? ? ? ?abstract_names.extend(get_abstract_names_for_item(item))

Nope, inlining that part makes sense.

>> ? ? ? for base in bases:
>> ? ? ? ? ? for name in getattr(base, "__abstractmethods__", ()):
>> ? ? ? ? ? ? ? # CHANGE 4: Using rpartition better tolerates weird
>> naming in the metaclass
>> ? ? ? ? ? ? ? # (weird naming in descriptors will still blow up in
>> the earlier search for abstract names)
>
> Could you provide an example of weird naming?

>>> class C(object):
...   pass
...
>>> setattr(C, 'weird.name', staticmethod(int))
>>> c = C()
>>> c.weird.name
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'C' object has no attribute 'weird'
>>> c.weird.name
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'C' object has no attribute 'weird'
>>> getattr(c, 'weird.name')()
0

This is definitely something that could legitimately be dismissed as
"well, don't do that then" (particularly since similarly weird names
on the descriptors will still break). However, I also prefer the way
partition based code reads over split-based code, so I still like the
modified version.

Full tolerance for weird naming would require storing 2-tuples in
__abstractmethods__ which would cause a whole new set of problems and
isn't worth the hassle.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From eliben at gmail.com  Thu Jun  9 06:30:57 2011
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 9 Jun 2011 07:30:57 +0300
Subject: [Python-Dev] The socket HOWTO
In-Reply-To: <BANLkTimVNHVZJPmi4Ex99QOwx6tWF8ETzA@mail.gmail.com>
References: <20110521170725.51eab5f9@pitrou.net> <4DE2643D.3070906@v.loewis.de>
	<20110604205026.374c0019@pitrou.net> <4DEB2306.5080206@v.loewis.de>
	<1F2B102E-29F8-4F12-801C-C3EE9170F08B@twistedmatrix.com>
	<4DEC04CA.1060608@v.loewis.de>
	<1E59F13B-852E-4DB7-8FB2-D7B067C90920@twistedmatrix.com>
	<BANLkTikdhh_rGk-Erh+NjUEMkSp5Th=NEQ@mail.gmail.com>
	<BANLkTimVNHVZJPmi4Ex99QOwx6tWF8ETzA@mail.gmail.com>
Message-ID: <BANLkTimS47aEh5kuA1_jaTMzyfdzNRX59g@mail.gmail.com>

On Wed, Jun 8, 2011 at 21:07, geremy condra <debatem1 at gmail.com> wrote:

> On Tue, Jun 7, 2011 at 10:37 AM, Eli Bendersky <eliben at gmail.com> wrote:
> >
> > Just be careful not to reproduce http://www.apress.com/9781590593714 :-)
> > These things tend to get out of hand very quickly.
>
> You say that like it's a bad thing. The first few chapters of that
> would make a great replacement for the howto.
>
>
Not a bad thing at all, and I'm sorry if I made it sound that way. I just
meant that it may turn into a *whole book* if too many details are added. I
had no intention to criticize this specific book. Frankly I didn't even read
it, I just remembered that a book with this title came out recently.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110609/9291eaf7/attachment.html>

From g.brandl at gmx.net  Thu Jun  9 08:16:05 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 09 Jun 2011 08:16:05 +0200
Subject: [Python-Dev] cpython (3.2): Fix #11583. Changed os.path.isdir
 to use GetFileAttributes instead of os.stat.
In-Reply-To: <E1QUSfj-0005Oq-79@dinsdale.python.org>
References: <E1QUSfj-0005Oq-79@dinsdale.python.org>
Message-ID: <ispof3$u9l$2@dough.gmane.org>

On 06/09/11 02:00, brian.curtin wrote:
> http://hg.python.org/cpython/rev/88e318166eaf
> changeset:   70713:88e318166eaf
> branch:      3.2
> parent:      70700:0aa3064d1cef
> user:        Brian Curtin <brian at python.org>
> date:        Wed Jun 08 18:17:18 2011 -0500
> summary:
>   Fix #11583. Changed os.path.isdir to use GetFileAttributes instead of os.stat.
> 
> By changing to the Windows GetFileAttributes API in nt._isdir we can figure
> out if the path is a directory without opening the file via os.stat. This has
> the minor benefit of speeding up os.path.isdir by at least 2x for regular
> files and 10-15x improvements were seen on symbolic links (which opened the
> file multiple times during os.stat). Since os.path.isdir is used in
> several places on interpreter startup, we get a minor speedup in startup time.
> 
> files:
>   Lib/ntpath.py         |  13 ++++++++++
>   Misc/NEWS             |   3 ++
>   Modules/posixmodule.c |  37 +++++++++++++++++++++++++++++++
>   3 files changed, 53 insertions(+), 0 deletions(-)
> 
> 
> diff --git a/Lib/ntpath.py b/Lib/ntpath.py
> --- a/Lib/ntpath.py
> +++ b/Lib/ntpath.py
> @@ -672,3 +672,16 @@
>  def sameopenfile(f1, f2):
>      """Test whether two file objects reference the same file"""
>      return _getfileinformation(f1) == _getfileinformation(f2)
> +
> +
> +try:
> +    # The genericpath.isdir implementation uses os.stat and checks the mode
> +    # attribute to tell whether or not the path is a directory.
> +    # This is overkill on Windows - just pass the path to GetFileAttributes
> +    # and check the attribute from there.
> +    from nt import _isdir
> +except ImportError:
> +    from genericpath import isdir as _isdir
> +
> +def isdir(path):
> +    return _isdir(path)

Not that it matters, but ISTM that this would be faster as

try:
    from nt import _isdir as isdir
except ImportError:
    pass

Georg


From victor.stinner at haypocalc.com  Thu Jun  9 11:05:17 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 09 Jun 2011 11:05:17 +0200
Subject: [Python-Dev] cpython (3.2): Fix #11583. Changed os.path.isdir
 to use GetFileAttributes instead of os.stat.
In-Reply-To: <ispof3$u9l$2@dough.gmane.org>
References: <E1QUSfj-0005Oq-79@dinsdale.python.org>
	<ispof3$u9l$2@dough.gmane.org>
Message-ID: <1307610317.15909.12.camel@marge>

Le jeudi 09 juin 2011 ? 08:16 +0200, Georg Brandl a ?crit :
> On 06/09/11 02:00, brian.curtin wrote:
> > http://hg.python.org/cpython/rev/88e318166eaf
> > changeset:   70713:88e318166eaf
> > branch:      3.2
> > parent:      70700:0aa3064d1cef
> > user:        Brian Curtin <brian at python.org>
> > date:        Wed Jun 08 18:17:18 2011 -0500
> > summary:
> >   Fix #11583. Changed os.path.isdir to use GetFileAttributes instead of os.stat.
> > 
> > By changing to the Windows GetFileAttributes API in nt._isdir we can figure
> > out if the path is a directory without opening the file via os.stat. This has
> > the minor benefit of speeding up os.path.isdir by at least 2x for regular
> > files and 10-15x improvements were seen on symbolic links (which opened the
> > file multiple times during os.stat). Since os.path.isdir is used in
> > several places on interpreter startup, we get a minor speedup in startup time.
> > 
> > files:
> >   Lib/ntpath.py         |  13 ++++++++++
> >   Misc/NEWS             |   3 ++
> >   Modules/posixmodule.c |  37 +++++++++++++++++++++++++++++++
> >   3 files changed, 53 insertions(+), 0 deletions(-)
> > 
> > 
> > diff --git a/Lib/ntpath.py b/Lib/ntpath.py
> > --- a/Lib/ntpath.py
> > +++ b/Lib/ntpath.py
> > @@ -672,3 +672,16 @@
> >  def sameopenfile(f1, f2):
> >      """Test whether two file objects reference the same file"""
> >      return _getfileinformation(f1) == _getfileinformation(f2)
> > +
> > +
> > +try:
> > +    # The genericpath.isdir implementation uses os.stat and checks the mode
> > +    # attribute to tell whether or not the path is a directory.
> > +    # This is overkill on Windows - just pass the path to GetFileAttributes
> > +    # and check the attribute from there.
> > +    from nt import _isdir
> > +except ImportError:
> > +    from genericpath import isdir as _isdir
> > +
> > +def isdir(path):
> > +    return _isdir(path)
> 
> Not that it matters, but ISTM that this would be faster as
> 
> try:
>     from nt import _isdir as isdir
> except ImportError:
>     pass

I would matter if _isdir() had a docstring, but it doesn't :-)
genericpath.isdir() has the following doc:

def isdir(s):
    """Return true if the pathname refers to an existing directory."""

Victor


From vinay_sajip at yahoo.co.uk  Thu Jun  9 12:51:31 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 9 Jun 2011 10:51:31 +0000 (UTC)
Subject: [Python-Dev] 3.2.1 and Issue 12291
Message-ID: <loom.20110609T124527-738@post.gmane.org>

I just filed an issue which shows a serious breakage in the marshal module:

"file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3"

http://bugs.python.org/issue12291

Perhaps this issue should be marked as a release blocker for 3.2.1 - what do
others (particularly Georg, of course) think? While it does appear to have been
broken for some time, it seems a bit too serious to leave until a 3.2.2 release.

Regards,

Vinay Sajip


From dsdale24 at gmail.com  Thu Jun  9 13:45:57 2011
From: dsdale24 at gmail.com (Darren Dale)
Date: Thu, 9 Jun 2011 07:45:57 -0400
Subject: [Python-Dev] Can we improve support for abstract base classes
 with desciptors
In-Reply-To: <BANLkTi=+0z8N70d7xnVGo+5=BCCOQphjHA@mail.gmail.com>
References: <BANLkTi=A_VuqQpNr+2VowEXLRENC7p-z2Q@mail.gmail.com>
	<BANLkTimPwrsRqzaAnknc-y9Qe1=ke3Gp9A@mail.gmail.com>
	<BANLkTinJLsE0whSoP4iQWhLrjS1GBMpdYg@mail.gmail.com>
	<BANLkTi=+0z8N70d7xnVGo+5=BCCOQphjHA@mail.gmail.com>
Message-ID: <BANLkTi=YAqRcr2ZuDqnVVS2cb+tt-8atog@mail.gmail.com>

On Wed, Jun 8, 2011 at 10:01 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Thu, Jun 9, 2011 at 8:51 AM, Darren Dale <dsdale24 at gmail.com> wrote:
>>> ? ? ? for base in bases:
>>> ? ? ? ? ? for name in getattr(base, "__abstractmethods__", ()):
>>> ? ? ? ? ? ? ? # CHANGE 4: Using rpartition better tolerates weird
>>> naming in the metaclass
>>> ? ? ? ? ? ? ? # (weird naming in descriptors will still blow up in
>>> the earlier search for abstract names)
>>
>> Could you provide an example of weird naming?
>
>>>> class C(object):
> ... ? pass
> ...
>>>> setattr(C, 'weird.name', staticmethod(int))
[...]

> This is definitely something that could legitimately be dismissed as
> "well, don't do that then" (particularly since similarly weird names
> on the descriptors will still break). However, I also prefer the way
> partition based code reads over split-based code, so I still like the
> modified version.

Yes, I like your modified version as well. I just wanted to understand
your concern, since it had never occurred to me to try something like
"setattr(C, 'pathological.name', ...)".

> Full tolerance for weird naming would require storing 2-tuples in
> __abstractmethods__ which would cause a whole new set of problems and
> isn't worth the hassle.

I'm glad you feel that way.

I'll work on a patch that includes docs and unit tests and post it at
http://bugs.python.org/issue11610. What do you think about deprecating
abstractproperty, or removing it from the documentation?

Darren

From ncoghlan at gmail.com  Thu Jun  9 14:27:05 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 9 Jun 2011 22:27:05 +1000
Subject: [Python-Dev] Can we improve support for abstract base classes
 with desciptors
In-Reply-To: <BANLkTi=YAqRcr2ZuDqnVVS2cb+tt-8atog@mail.gmail.com>
References: <BANLkTi=A_VuqQpNr+2VowEXLRENC7p-z2Q@mail.gmail.com>
	<BANLkTimPwrsRqzaAnknc-y9Qe1=ke3Gp9A@mail.gmail.com>
	<BANLkTinJLsE0whSoP4iQWhLrjS1GBMpdYg@mail.gmail.com>
	<BANLkTi=+0z8N70d7xnVGo+5=BCCOQphjHA@mail.gmail.com>
	<BANLkTi=YAqRcr2ZuDqnVVS2cb+tt-8atog@mail.gmail.com>
Message-ID: <BANLkTi=YdwFibC-0EsXCoNCJfL6NZcy-dw@mail.gmail.com>

On Thu, Jun 9, 2011 at 9:45 PM, Darren Dale <dsdale24 at gmail.com> wrote:
> What do you think about deprecating
> abstractproperty, or removing it from the documentation?

Unless anyone specifically howls at the idea, +1 to both (since
abstractproperty doesn't actually achieve the intended purpose and
will become redundant with the proper fix in 3.3).

Cheers,
Nick.


-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From barry at python.org  Thu Jun  9 17:01:42 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 9 Jun 2011 11:01:42 -0400
Subject: [Python-Dev] 3.2.1 and Issue 12291
In-Reply-To: <loom.20110609T124527-738@post.gmane.org>
References: <loom.20110609T124527-738@post.gmane.org>
Message-ID: <20110609110142.62cdd2eb@neurotica.wooz.org>

On Jun 09, 2011, at 10:51 AM, Vinay Sajip wrote:

>I just filed an issue which shows a serious breakage in the marshal module:
>
>"file written using marshal in 3.2 can be read by 2.7, but not 3.2 or 3.3"
>
>http://bugs.python.org/issue12291
>
>Perhaps this issue should be marked as a release blocker for 3.2.1 - what do
>others (particularly Georg, of course) think? While it does appear to have
>been broken for some time, it seems a bit too serious to leave until a 3.2.2
>release.

Sounds bad ;).  I marked it a release blocker, but of course Georg will have
the final say.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110609/202fde78/attachment.pgp>

From brian.curtin at gmail.com  Thu Jun  9 17:04:58 2011
From: brian.curtin at gmail.com (Brian Curtin)
Date: Thu, 9 Jun 2011 10:04:58 -0500
Subject: [Python-Dev] cpython (3.2): Fix #11583. Changed os.path.isdir
 to use GetFileAttributes instead of os.stat.
In-Reply-To: <1307610317.15909.12.camel@marge>
References: <E1QUSfj-0005Oq-79@dinsdale.python.org>
	<ispof3$u9l$2@dough.gmane.org> <1307610317.15909.12.camel@marge>
Message-ID: <BANLkTinOP_tju6ouCdevt8NZHZXCY+FO=A@mail.gmail.com>

On Thu, Jun 9, 2011 at 04:05, Victor Stinner
<victor.stinner at haypocalc.com>wrote:

> Le jeudi 09 juin 2011 ? 08:16 +0200, Georg Brandl a ?crit :
> > On 06/09/11 02:00, brian.curtin wrote:
> > > http://hg.python.org/cpython/rev/88e318166eaf
> > > changeset:   70713:88e318166eaf
> > > branch:      3.2
> > > parent:      70700:0aa3064d1cef
> > > user:        Brian Curtin <brian at python.org>
> > > date:        Wed Jun 08 18:17:18 2011 -0500
> > > summary:
> > >   Fix #11583. Changed os.path.isdir to use GetFileAttributes instead of
> os.stat.
> > >
> > > By changing to the Windows GetFileAttributes API in nt._isdir we can
> figure
> > > out if the path is a directory without opening the file via os.stat.
> This has
> > > the minor benefit of speeding up os.path.isdir by at least 2x for
> regular
> > > files and 10-15x improvements were seen on symbolic links (which opened
> the
> > > file multiple times during os.stat). Since os.path.isdir is used in
> > > several places on interpreter startup, we get a minor speedup in
> startup time.
> > >
> > > files:
> > >   Lib/ntpath.py         |  13 ++++++++++
> > >   Misc/NEWS             |   3 ++
> > >   Modules/posixmodule.c |  37 +++++++++++++++++++++++++++++++
> > >   3 files changed, 53 insertions(+), 0 deletions(-)
> > >
> > >
> > > diff --git a/Lib/ntpath.py b/Lib/ntpath.py
> > > --- a/Lib/ntpath.py
> > > +++ b/Lib/ntpath.py
> > > @@ -672,3 +672,16 @@
> > >  def sameopenfile(f1, f2):
> > >      """Test whether two file objects reference the same file"""
> > >      return _getfileinformation(f1) == _getfileinformation(f2)
> > > +
> > > +
> > > +try:
> > > +    # The genericpath.isdir implementation uses os.stat and checks the
> mode
> > > +    # attribute to tell whether or not the path is a directory.
> > > +    # This is overkill on Windows - just pass the path to
> GetFileAttributes
> > > +    # and check the attribute from there.
> > > +    from nt import _isdir
> > > +except ImportError:
> > > +    from genericpath import isdir as _isdir
> > > +
> > > +def isdir(path):
> > > +    return _isdir(path)
> >
> > Not that it matters, but ISTM that this would be faster as
> >
> > try:
> >     from nt import _isdir as isdir
> > except ImportError:
> >     pass
>
> I would matter if _isdir() had a docstring, but it doesn't :-)
> genericpath.isdir() has the following doc:
>
> def isdir(s):
>    """Return true if the pathname refers to an existing directory."""


http://hg.python.org/lookup/d40609dd01e0 adds the docstring back in and
redoes the imports as Georg mentioned, which is better.
Thanks for having a look.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110609/03852942/attachment.html>

From status at bugs.python.org  Fri Jun 10 18:07:22 2011
From: status at bugs.python.org (Python tracker)
Date: Fri, 10 Jun 2011 18:07:22 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20110610160722.E6A101CDA4@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2011-06-03 - 2011-06-10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2826 (+11)
  closed 21268 (+47)
  total  24094 (+58)

Open issues with patches: 1236 


Issues opened (45)
==================

#5906: Risk of confusion in multiprocessing module - daemonic process
http://bugs.python.org/issue5906  reopened by pakal

#9516: sysconfig: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.3" but 
http://bugs.python.org/issue9516  reopened by eric.araujo

#12255: A few changes to .*ignore
http://bugs.python.org/issue12255  opened by eric.araujo

#12256: Link isinstance/issubclass doc to abc module
http://bugs.python.org/issue12256  opened by eric.araujo

#12257: Rework/replace use of DISTUTILS_USE_SDK in packaging
http://bugs.python.org/issue12257  opened by eric.araujo

#12258: Clean up bytes I/O in get_compiler_versions
http://bugs.python.org/issue12258  opened by eric.araujo

#12259: Test and document which compilers can be created on which plat
http://bugs.python.org/issue12259  opened by eric.araujo

#12260: Make install default to user site-packages
http://bugs.python.org/issue12260  opened by eric.araujo

#12261: urllib.parse docs still refer to urlparse
http://bugs.python.org/issue12261  opened by tlesher

#12263: punycode codec ignores the error handler argument
http://bugs.python.org/issue12263  opened by haypo

#12268: file readline, readlines & readall methods can lose data on EI
http://bugs.python.org/issue12268  opened by gregory.p.smith

#12271: Python 2.7.x on IA64 running SLES 11 SP1
http://bugs.python.org/issue12271  opened by v.claudic

#12272: Python 2.7.1 version conflict for package "Tcl" on Windows 7
http://bugs.python.org/issue12272  opened by jackie3

#12273: Change ast.__version__ calculation	to provide consistent order
http://bugs.python.org/issue12273  opened by ncoghlan

#12274: "Print window" menu on IDLE aborts whole application
http://bugs.python.org/issue12274  opened by gagenellina

#12275: urllib.request.HTTPRedirectHandler won't redirect to a URL wit
http://bugs.python.org/issue12275  opened by lilydjwg

#12276: 3.x ignores sys.tracebacklimit=0
http://bugs.python.org/issue12276  opened by gagenellina

#12277: Missing comma in os.walk docs
http://bugs.python.org/issue12277  opened by Retro

#12278: Core mentorship mention in the devguide
http://bugs.python.org/issue12278  opened by adam.woodbeck

#12279: Add build_distinfo command to packaging
http://bugs.python.org/issue12279  opened by eric.araujo

#12281: bytes.decode('mbcs', 'ignore') does replace undecodable bytes 
http://bugs.python.org/issue12281  opened by haypo

#12282: build process adds CWD (null entry) to PYTHONPATH if PYTHONPAT
http://bugs.python.org/issue12282  opened by r.david.murray

#12284: argparse.ArgumentParser: usage example option
http://bugs.python.org/issue12284  opened by jonash

#12285: Unexpected behavior for 0 or negative processes in multiproces
http://bugs.python.org/issue12285  opened by jorgsk

#12287: ossaudiodev: stack corruption with FD >= FD_SETSIZE
http://bugs.python.org/issue12287  opened by neologix

#12288: Python 2.7.1 tkSimpleDialog initialvalue
http://bugs.python.org/issue12288  opened by busfault

#12289: http.server.CGIHTTPRequestHandler doesn't check if a Python sc
http://bugs.python.org/issue12289  opened by haypo

#12290: __setstate__ is called for false values
http://bugs.python.org/issue12290  opened by eltoder

#12291: file written using marshal in 3.2 can be read by 2.7, but not 
http://bugs.python.org/issue12291  opened by vinay.sajip

#12294: multiprocessing.Pool: Need a way to find out if work are finis
http://bugs.python.org/issue12294  opened by mozbugbox

#12295: Fix ResourceWarning in turtledemo help window
http://bugs.python.org/issue12295  opened by eric.araujo

#12296: Minor clarification in devguide
http://bugs.python.org/issue12296  opened by eric.araujo

#12297: Clarifications to atexit.register and unregister doc
http://bugs.python.org/issue12297  opened by eric.araujo

#12299: Stop documenting functions added by site as builtins
http://bugs.python.org/issue12299  opened by eric.araujo

#12300: Document pydoc.help
http://bugs.python.org/issue12300  opened by eric.araujo

#12301: Use :data:`sys.thing` instead of ``sys.thing`` throughout
http://bugs.python.org/issue12301  opened by eric.araujo

#12302: packaging test command needs access to .dist-info
http://bugs.python.org/issue12302  opened by michael.mulich

#12303: expose sigwaitinfo() and sigtimedwait() in the signal module
http://bugs.python.org/issue12303  opened by haypo

#12304: expose signalfd(2) in the signal module
http://bugs.python.org/issue12304  opened by haypo

#12305: Building PEPs doesn't work on Python 3
http://bugs.python.org/issue12305  opened by ericsnow

#12306: zlib: Expose zlibVersion to query runtime version of zlib
http://bugs.python.org/issue12306  opened by torsten

#12307: Inconsistent formatting of section titles in PEP 0
http://bugs.python.org/issue12307  opened by ericsnow

#12308: Add link to PEP 0 for topical index in wiki
http://bugs.python.org/issue12308  opened by ericsnow

#12309: os.environ was modified by test_packaging
http://bugs.python.org/issue12309  opened by haypo

#12310: Segfault in test_multiprocessing
http://bugs.python.org/issue12310  opened by haypo



Most recent 15 issues with no replies (15)
==========================================

#12309: os.environ was modified by test_packaging
http://bugs.python.org/issue12309

#12308: Add link to PEP 0 for topical index in wiki
http://bugs.python.org/issue12308

#12307: Inconsistent formatting of section titles in PEP 0
http://bugs.python.org/issue12307

#12306: zlib: Expose zlibVersion to query runtime version of zlib
http://bugs.python.org/issue12306

#12305: Building PEPs doesn't work on Python 3
http://bugs.python.org/issue12305

#12303: expose sigwaitinfo() and sigtimedwait() in the signal module
http://bugs.python.org/issue12303

#12301: Use :data:`sys.thing` instead of ``sys.thing`` throughout
http://bugs.python.org/issue12301

#12300: Document pydoc.help
http://bugs.python.org/issue12300

#12299: Stop documenting functions added by site as builtins
http://bugs.python.org/issue12299

#12297: Clarifications to atexit.register and unregister doc
http://bugs.python.org/issue12297

#12296: Minor clarification in devguide
http://bugs.python.org/issue12296

#12295: Fix ResourceWarning in turtledemo help window
http://bugs.python.org/issue12295

#12294: multiprocessing.Pool: Need a way to find out if work are finis
http://bugs.python.org/issue12294

#12288: Python 2.7.1 tkSimpleDialog initialvalue
http://bugs.python.org/issue12288

#12272: Python 2.7.1 version conflict for package "Tcl" on Windows 7
http://bugs.python.org/issue12272



Most recent 15 issues waiting for review (15)
=============================================

#12308: Add link to PEP 0 for topical index in wiki
http://bugs.python.org/issue12308

#12306: zlib: Expose zlibVersion to query runtime version of zlib
http://bugs.python.org/issue12306

#12304: expose signalfd(2) in the signal module
http://bugs.python.org/issue12304

#12297: Clarifications to atexit.register and unregister doc
http://bugs.python.org/issue12297

#12296: Minor clarification in devguide
http://bugs.python.org/issue12296

#12295: Fix ResourceWarning in turtledemo help window
http://bugs.python.org/issue12295

#12291: file written using marshal in 3.2 can be read by 2.7, but not 
http://bugs.python.org/issue12291

#12289: http.server.CGIHTTPRequestHandler doesn't check if a Python sc
http://bugs.python.org/issue12289

#12287: ossaudiodev: stack corruption with FD >= FD_SETSIZE
http://bugs.python.org/issue12287

#12285: Unexpected behavior for 0 or negative processes in multiproces
http://bugs.python.org/issue12285

#12281: bytes.decode('mbcs', 'ignore') does replace undecodable bytes 
http://bugs.python.org/issue12281

#12278: Core mentorship mention in the devguide
http://bugs.python.org/issue12278

#12277: Missing comma in os.walk docs
http://bugs.python.org/issue12277

#12268: file readline, readlines & readall methods can lose data on EI
http://bugs.python.org/issue12268

#12261: urllib.parse docs still refer to urlparse
http://bugs.python.org/issue12261



Top 10 most discussed issues (10)
=================================

#12248: __dir__ semantics changed in Python 2.7.2
http://bugs.python.org/issue12248  18 msgs

#12014: str.format parses replacement field incorrectly
http://bugs.python.org/issue12014  12 msgs

#12246: Warn when trying to install third-party module from an uninsta
http://bugs.python.org/issue12246  12 msgs

#8927: Handle version incompatibilities in dependencies
http://bugs.python.org/issue8927  11 msgs

#12211: Better document math.copysign behavior.
http://bugs.python.org/issue12211  11 msgs

#12226: use secured channel for uploading packages to pypi
http://bugs.python.org/issue12226  11 msgs

#11457: Expose nanosecond precision from system calls
http://bugs.python.org/issue11457  10 msgs

#7511: msvc9compiler.py: ValueError when trying to compile with VC Ex
http://bugs.python.org/issue7511   8 msgs

#12084: os.stat() on windows doesn't consider relative symlink
http://bugs.python.org/issue12084   8 msgs

#12291: file written using marshal in 3.2 can be read by 2.7, but not 
http://bugs.python.org/issue12291   8 msgs



Issues closed (47)
==================

#3771: test_httpservers intermittent failure, test_post and EINTR
http://bugs.python.org/issue3771  closed by gregory.p.smith

#3992: remove custom log module from distutils2
http://bugs.python.org/issue3992  closed by eric.araujo

#4949: Constness in PyErr_NewException
http://bugs.python.org/issue4949  closed by benjamin.peterson

#7015: Getting call trace while executing "modules spam" at help prom
http://bugs.python.org/issue7015  closed by eric.araujo

#8407: expose pthread_sigmask(), pthread_kill(), sigpending() and sig
http://bugs.python.org/issue8407  closed by haypo

#9205: Parent process hanging in multiprocessing if children terminat
http://bugs.python.org/issue9205  closed by pitrou

#9344: please add posix.getgrouplist()
http://bugs.python.org/issue9344  closed by rosslagerwall

#10027: os.lstat/os.stat don't set st_nlink on Windows
http://bugs.python.org/issue10027  closed by brian.curtin

#10424: better error message from argparse when positionals missing
http://bugs.python.org/issue10424  closed by r.david.murray

#10645: Remove egg-info files in stdlib
http://bugs.python.org/issue10645  closed by eric.araujo

#10694: zipfile.py end of central directory detection not robust
http://bugs.python.org/issue10694  closed by r.david.murray

#10884: pkgutil EggInfoDistribution requirements for .egg-info metadat
http://bugs.python.org/issue10884  closed by michael.mulich

#11028: Implement the setup.py -> setup.cfg in mkcfg
http://bugs.python.org/issue11028  closed by eric.araujo

#11041: On the distutils2 documentation, 'requires-python' shouldn't b
http://bugs.python.org/issue11041  closed by eric.araujo

#11092: setup.cfg isn't packaged when running sdist
http://bugs.python.org/issue11092  closed by eric.araujo

#11203: gzip doc is behind
http://bugs.python.org/issue11203  closed by teamnoir

#11583: os.path.isdir() is slow on windows
http://bugs.python.org/issue11583  closed by brian.curtin

#11893: Obsolete SSLFakeFile in smtplib?
http://bugs.python.org/issue11893  closed by pitrou

#12019: Dead or buggy code in importlib.test.__main__
http://bugs.python.org/issue12019  closed by eric.araujo

#12021: mmap.read requires an argument
http://bugs.python.org/issue12021  closed by neologix

#12040: Expose a Process.sentinel property (and fix polling loop in Pr
http://bugs.python.org/issue12040  closed by pitrou

#12080: decimal.py: performance in _power_exact
http://bugs.python.org/issue12080  closed by mark.dickinson

#12127: Inconsistent leading zero treatment
http://bugs.python.org/issue12127  closed by mark.dickinson

#12168: SysLogHandler incorrectly appends \000 to messages
http://bugs.python.org/issue12168  closed by vinay.sajip

#12214: platform module can't detect archlinux distribution
http://bugs.python.org/issue12214  closed by eric.araujo

#12224: problem with siginterrupt
http://bugs.python.org/issue12224  closed by haypo

#12234: unittest2 could enable regex debugging for more information
http://bugs.python.org/issue12234  closed by r.david.murray

#12237: Document how to open non-default webbrowser
http://bugs.python.org/issue12237  closed by terry.reedy

#12249: add missing command
http://bugs.python.org/issue12249  closed by eric.araujo

#12254: PEP-3107 has a wrong attribute name for function annotations
http://bugs.python.org/issue12254  closed by eric.araujo

#12262: Not Inheriting File Descriptors on Windows?
http://bugs.python.org/issue12262  closed by 3vi1

#12264: parser.expr(): reference count error if more than one argument
http://bugs.python.org/issue12264  closed by python-dev

#12265: revamp argument errors
http://bugs.python.org/issue12265  closed by python-dev

#12266: str.capitalize contradicts oneself
http://bugs.python.org/issue12266  closed by r.david.murray

#12267: Difficult to compile python in Visual Studio 2010 express
http://bugs.python.org/issue12267  closed by loewis

#12269: IDLE 2.7 hangs on Mac OS X 10.6
http://bugs.python.org/issue12269  closed by ned.deily

#12270: Please update copyright notice on bugs.python.org to 2011?
http://bugs.python.org/issue12270  closed by ezio.melotti

#12280: If statement
http://bugs.python.org/issue12280  closed by brian.curtin

#12283: python3.2 smtplib _quote_periods
http://bugs.python.org/issue12283  closed by r.david.murray

#12286: float('nan') breaks sort() method on a list of floats
http://bugs.python.org/issue12286  closed by ezio.melotti

#12292: bpython
http://bugs.python.org/issue12292  closed by amaury.forgeotdarc

#12293: wrong arguments passed to SMTP.sendmail in example
http://bugs.python.org/issue12293  closed by r.david.murray

#12298: Sphinx glitch in library/functions
http://bugs.python.org/issue12298  closed by georg.brandl

#12311: memory leak with self-referencing dict
http://bugs.python.org/issue12311  closed by Albert.Zeyer

#12312: is ok
http://bugs.python.org/issue12312  closed by brian.curtin

#1083299: Distutils doesn't pick up all the files it should.
http://bugs.python.org/issue1083299  closed by eric.araujo

#1699259: replacing char* with const char* in sysmodule.c/.h
http://bugs.python.org/issue1699259  closed by sebastinas

From guido at python.org  Fri Jun 10 21:49:26 2011
From: guido at python.org (Guido van Rossum)
Date: Fri, 10 Jun 2011 12:49:26 -0700
Subject: [Python-Dev] cpython: Remove some extraneous parentheses and
 swap the comparison order to
In-Reply-To: <BANLkTimrqQ3VfCkpBa8HvPeB+igTOLB06w@mail.gmail.com>
References: <E1QTmqU-0004iT-VX@dinsdale.python.org>
	<iski44$q0m$1@dough.gmane.org>
	<4DEDE95A.2010603@egenix.com> <1307482529.16924.331.camel@surprise>
	<BANLkTimrqQ3VfCkpBa8HvPeB+igTOLB06w@mail.gmail.com>
Message-ID: <BANLkTi=WaEx6stwceHs=Y_iQb1RuKP0o8Q@mail.gmail.com>

On Wed, Jun 8, 2011 at 8:12 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Jun 8, 2011 at 7:35 AM, David Malcolm <dmalcolm at redhat.com> wrote:
>> After ~12 years of doing this, it comes naturally. ?I appreciate that
>> this may come across as weird though :)
>
> I actually thought Brett's rationale in the checkin comment was
> reasonable (if you get in the habit of putting constants on the left,
> then the classic "'=' instead of '=='" typo is a compiler error
> instead of a reassignment).

I really like consistency across the code base. I really don't like
constant-on-the-left, and it's basically not used in the current
codebase. Please be consistent and don't start using it.

> Call it a +0 in favour of letting people put constants on the left in
> C code if they prefer it that way, so long as any given if/elif chain
> is consistent in the style it uses.

Sorry, I give it a -1. (I'd like to be able to read the codebase still... :-)

-- 
--Guido van Rossum (python.org/~guido)

From wolfson at gmail.com  Fri Jun 10 23:15:54 2011
From: wolfson at gmail.com (Ben Wolfson)
Date: Fri, 10 Jun 2011 14:15:54 -0700
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
Message-ID: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>

Hello,

I'm writing because discussion in a bug report I submitted
(<http://bugs.python.org/issue12014>) has suggested that, insofar as
at least part of the issue revolves around the interpretation of PEP
3101, that aspect belonged on python-dev. In particular, I was told
that the PEP, not the documentation, is authoritative. Since I'm the
one who thinks something is wrong, it seems appropriate for me to be
the one to bring it up.

Basically, the issue is that the current behavior of str.format is at
variance at least with the documentation
<http://docs.python.org/library/string.html#formatstrings>, is almost
certainly at variance with PEP3101 in one respect, and is in my
opinion at variance with PEP3101 in another respect as well, regarding
what characters can be present in what the grammar given in the
documentation calls an element_index, that is, the bit between the
square brackets in "{0.attr[idx]}".

Both discovering the current behavior and interpreting the
documentation are pretty straightforward; interpreting what the PEP
actually calls for is more vexed. I'll do the first two things first.
TOC for the remainder:

1. What does the current implementation do?
2. What does the documentation say?
3. What does the PEP say? [this part is long, but the PEP is not
clear, and I wanted to be thorough]
4. Who cares?

1. What does the current implementation do?

Suppose you have this dictionary:

d = {"@": 0,
     "!": 1,
     ":": 2,
     "^": 3,
     "}": 4,
     "{": {"}": 5},
    }

Then the following expressions have the following results:

(a) "{0[@]}".format(d)    --> '0'
(b) "{0[!]}".format(d)    --> ValueError: Missing ']' in format string
(c) "{0[:]}".format(d)    --> ValueError: Missing ']' in format string
(d) "{0[^]}".format(d)    --> '3'
(e) "{0[}]}".format(d)    --> ValueError: Missing ']' in format string
(f) "{0[{]}".format(d)    --> ValueError: unmatched '{' in format
(g) "{0[{][}]}".format(d) --> '5'

Given (e) and (f), I think (g) should be a little surprising, though
you can probably guess what's going on and it's not hard to see why it
happens if you look at the source: (e) and (f) fail because
MarkupIterator_next (in Objects/stringlib/string_format.h) scans
through the string looking for curly braces, because it treats them as
semantically significant no matter what context they occur in. So,
according to MarkupIterator_next, the *first* right curly brace in (e)
indicates the end of the replacement field, giving "{0[}". In (f), the
second left curly brace indicates (to MarkupIterator_next) the start
of a *new* replacement field, and since there's only one right curly
brace, it complains. In (g), MarkupIterator_next treats the second
left curly brace as starting a new replacement field and the first
right curly brace as closing it. However, actually, those braces don't
define new replacement fields, as indicated by the fact that the whole
expression treats the element_index fields as just plain old strings.
(So the current implementation is somewhat schizophrenic, acting at
one point as if the braces have to be balanced because '{[]}' is a
replacement field and at another point treating the braces as
insignificant.)

The explanation for (b) and (c) is that parse_field (same source file)
treats ':' and '!'  as indicating the end of the field_name section of
the replacement field, regardless of whether those characters occur
within square brackets or not.

So, that's what the current implementation does, in those cases.

2. What does the documentation say?

The documentation gives a grammar for replacement fields:

"""
replacement_field ::=  "{" [field_name] ["!" conversion] [":" format_spec] "}"
field_name        ::=  arg_name ("." attribute_name | "[" element_index "]")*
arg_name          ::=  [identifier | integer]
attribute_name    ::=  identifier
element_index     ::=  integer | index_string
index_string      ::=  <any source character except "]"> +
conversion        ::=  "r" | "s"
format_spec       ::=  <described in the next section>
"""

Given this definition of index_string, all of (a)--(g) should be
legal, and the results should be the strings '0', '1', '2', '3',
"{'}': 5}", and '5'. There is no need to exclude ':', '!', '}', or '{'
from the index_string field; allowing them creates no ambiguity,
because the field is delimited by square brackets.

Tangent: the definition of attribute_name here also does not describe
the current behavior ("{0.  ;}".format(x) works fine and will call
getattr(x, " ;")) and the PEP does not require the attribute_name to
be an identifier; in fact it explicitly states that the attribute_name
doesn't need to be a valid Python identifier. attribute_name should
read (to reflect something like actual behavior, anyway) "<any source
character except '[', '.', ':', '!', '{', or '}'> +". The same goes
for arg_name (with the integer alternation). Observe:

>>> x = lambda: None
>>> setattr(x, ']]', 3)
>>> "{].]]}".format(**{"]":x})     # (h)
'3'

One can also presently do this (hence "something like actual behavior"):
>>> setattr(x, 'f}', 4)
>>> "{a{s.f}}".format(**{"a{s":x})
'4'
But not this:
>>> "{a{s.func_name}".format(**{"a{s":x})
as it raises a ValueError, for the same reason as explains (g) above.


3. What does the PEP say?

Well... It's actually hard to tell!

Summary: The PEP does not contain a grammar for replacement fields,
and is surprisingly nonspecific about what can appear where, at least
when talking about the part of the replacement field before the format
specifier. The most reasonable interpretation of the parts of the PEP
that seem to be relevant favors the documentation, rather than the
implementation.

This can be separated into two sub-questions.

A. What does the PEP say about ':' and '!'?

It says two things that pertain to element_index fields.

The first is this:
"""
                   The rules for parsing an item key are very simple.
    If it starts with a digit, then it is treated as a number, otherwise
    it is used as a string.

    Because keys are not quote-delimited, it is not possible to
    specify arbitrary dictionary keys (e.g., the strings "10" or
    ":-]") from within a format string.
"""

So it notes that some things can't be used:

 - Because anything composed entirely of digits is treated as a
number, you can't get a string composed entirely of digits. Clear
enough.

 - What's the explanation for the second example, though? Well, you
can't have a right square bracket in the index_string, so that would
already mean that you can't do this: "{0[:-]]}".format(...) regardless
of the whether colons are legal or not. So, although the PEP gives an
example of a string that can't in the element_index part of a
replacement field, and that string contains a colon, that string would
have been disallowed for other reasons anyway.

The second is this:

"""
                                  The str.format() function will have
    a minimalist parser which only attempts to figure out when it is
    "done" with an identifier (by finding a '.' or a ']', or '}',
    etc.).
"""

This requires some interpretation. For one thing, the contents of an
element_index aren't identifiers. For another, it's not true that
you're done with an identifier (or whatever) whenever you see *any* of
those; it depends on context. When parsing this: "{0[a.b]}" the parser
should not stop at the '.'; it should keep going until it reaches the
']', and that will give the element_index. And when parsing this:
"{0.f]oo[bar].baz}", it's done with the identifier "foo" when it
reaches the '[', not when it reaches the second '.', and not when it
reaches the ']', either (recall (h)). The "minimalist parser" is, I
take it, one that works like this: when parsing an arg_name you're
done when you reach a '[', a ':', a '!', a '.', '{', or a '}'; the
same rules apply when parsing a attribute_name; when parsing an
element_index you're done when you see a ']'.

Now as regards the curly braces there are some other parts of the PEP
that perhaps should be taken into account (see below), but I can find
no justification for excluding ':' and '!' from the element_index
field; the bit quoted above about having a minimalist parser isn't a
good justification for that, and it's the only part of the entire PEP
that comes close to addressing the question.

B. What does it say about '}' and '{'?

It still doesn't say much explicitly. There's the quotation I just
gave, and then these passages:

"""
    Brace characters ('curly braces') are used to indicate a
    replacement field within the string:

    [...]

    Braces can be escaped by doubling:
"""

Taken by itself, this would suggest that wherever there's an unescaped
brace, there's a replacement field. That would mean that the current
implementation's behavior is correct in (e) and (f) but incorrect in
(g). However, I think this is a bad interpretation; unescaped braces
can indicate the presence of a replacement field without that meaning
that *within* a replacement field braces have that meaning, no matter
where within the replacement field they occur.

Later in the PEP, talking about this example:

        "{0:{1}}".format(a, b)

We have this:

"""
    These 'internal' replacement fields can only occur in the format
    specifier part of the replacement field.  Internal replacement fields
    cannot themselves have format specifiers.  This implies also that
    replacement fields cannot be nested to arbitrary levels.

    Note that the doubled '}' at the end, which would normally be
    escaped, is not escaped in this case.  The reason is because
    the '{{' and '}}' syntax for escapes is only applied when used
    *outside* of a format field.  Within a format field, the brace
    characters always have their normal meaning.
"""

The claim "within a format field, the brace characters always have
their normal meaning" might be taken to mean that within a
*replacement* field, brace characters always indicate the start (or
end) of a replacement field. But the PEP at this point is clearly
talking about the formatting section of a replacement field---the part
that follows the ':', if present. ("Format field" is nowhere defined
in the PEP, but it seems reasonable to take it to mean "the format
specifier of a replacement field".) However, it seems most reasonable
to me to take "normal meaning" to mean "just a brace character".

Note that the present implementation only kinda sorta conforms to the
PEP in this respect:


>>> import datetime
>>> format(datetime.datetime.now(), "{{%Y")
'{{2011'
>>> "{0:{{%{1}}".format(datetime.datetime.now(), 'Y') # (i)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmatched '{' in format
>>> "{0:{{%{1}}}}".format(datetime.datetime.now(), 'Y') # (j)
'{2011}'

Here the brace characters in (i) and (j) are treated, again in
MarkupIterator_next, as indicating the start of a replacement field.
In (i), this leads the function to throw an exception; since they're
balanced in (j), processing proceeds further, and the doubled braces
aren't treated as indicating the start or end of a replacement
field---because they're escaped.  Given that the format spec part of a
replacement field can contain further replacement fields, this is, I
think, correct behavior, but it's not easy to see how it comports with
the PEP, whose language is not very exact.

The call to the built-in format() bypasses the mechanism that leads to
these results.

The PEP is very, very nonspecific about the parts of the replacement
field that precede the format specifier. I don't know what kind of
discussion surrounded the drawing up of the grammar that appears in
the documentation, but I think that it, and not the implementation,
should be followed.

The implementation only works the way it does because of parsing
shortcuts: it raises ValueErrors for (b) and (c) because it
generalizes something true of the attribute_name field (encountering a
':' or '!' means one has moved on to the format_spec or conversion
part of the replacement field) to the element_index field. And it
raises an error for (e) and (f), but lets (g) through, for the reasons
already mentioned. It is, in that respect, inconsistent; it treats the
curly brace as having one semantic significance at one point and an
entirely different significance at another point, so that it does the
right thing in the case of (g) entirely by accident. There is, I
think, no way to construe the PEP so that it is reasonable to do what
the present implementation does in all three cases (if "{" indicates
the start of a replacement field in (f), it should do so in (g) as
well); I think it's
actually pretty difficult to construe the PEP in any way that makes
what it does in the case of (e) and (f) correct.

4. Who cares?

Well, I do. (Obviously.) I even have a use case: I want to be able to
put arbitrary (or as close to arbitrary as possible) strings in the
element_index field, because I've got some objects that (should!)
enable me to do this:

p.say("I'm warning you, {e.red.underline[don't touch that!]}")

and have this written ("e" for "effects") to p.out:

I'm warning you, \x1b[31m\x1b[4mdon't touch that!\x1b[0m

I have a way around the square bracket problem, but it would be quite
burdensome to have to deal with all of !:{} as well; enough that
I would fall back on something like this:

"I'm warning you, {0}".format(e.red.underline("don't touch that!"))

or some similar interpolation-based strategy, which I think would be a
shame, because of the way it chops up the string.

But I also think the present behavior is extremely counterintuitive,
unnecessarily complex, and illogical (even unpythonic!). Isn't it
bizarre that (g) should work, given what (e) and (f) do? Isn't it
strange that (b) and (c) should fail, given that there's no real
reason for them to do so---no ambiguity that has to be avoided? And
something's gotta give; the documentation and the implementation do
not correspond.

Beyond the counterintuitiveness of the present implementation, it is
also, I think, self-inconsistent. (e) and (f) fail because the
interior brace is treated as starting or ending a replacement field,
even though interior replacement fields aren't allowed in that part of
a replacement field. (g) succeeds because the braces are balanced:
they are treated at one point as if they were part of a replacement
field, and at another (correctly) as if they are not. But this makes
the failure of (e) and (f) unaccountable. It would not have been
impossible for the PEP to allow interior replacement fields anywhere,
and not just in the format spec, in which case we might have had this:

(g') "{0[{][}]}".format(range(10), **{'][':4}) --> '3'
or this:
(g'') "{0[{][}]}".format({'4':3}, **{'][':4}) --> '3'
or something with that general flavor.

As far as I can tell, the only way to consistently maintain that (e)
and (f) should fail requires that one take (g') or (g'') to be
correct: either the interior braces signal replacement fields (hence
must be balanced) or they don't (or they're escaped).

Regarding the documentation, it could of course be rendered correct by
changing *it*, rather than the implementation. But wouldn't it be
tremendously weird to have to explain that, in the part of the
replacement field preceding the conversion, you can't employ any curly
braces, unless they're balanced, and you can't employ ':' or '!' at
all, even though they have no semantic significance? So these are out:

{0[{].foo}
{0[}{}]}
{0[a:b]}

But these are in:

{0[{}{}]}
{0[{{].foo.}}}  (k)

((k) does work, if you give it an object with the right structure,
though it probably should not.)

And, moreover, someone would then have to actually change the
documentation, whereas there's a patch already, attached to the bug
report linked way up at the top of this email, that makes (a)--(g) all
work, leaves (i) and (j) as they are, and has the welcome side-effect
of making (k) *not* work (if any code anywhere relies on (k) or things
like it working, I will be very surprised: anyway the fact that (k)
works is, technically, undocumented). It is also quite simple. It
doesn't effect the built-in format(), because the built-in format() is
concerned only with the format-specifier part of the replacement field
and not all the stuff that comes before that, telling str.format
*what* object to format.

Thanks for reading,
-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]

From sam.edwards at Colorado.EDU  Sat Jun 11 02:23:47 2011
From: sam.edwards at Colorado.EDU (Sam Edwards)
Date: Fri, 10 Jun 2011 18:23:47 -0600
Subject: [Python-Dev] Python jails
Message-ID: <4DF2B593.8040302@colorado.edu>

Hello! This is my first posting to the python-dev list, so please
forgive me if I violate any unspoken etiquette here. :)

I was looking at Python 2.x's f_restricted frame flag (or, rather, the
numerous ways around it) and noticed that most (all?)
of the attacks to escape restricted execution involved the attacker
grabbing something he wasn't supposed to have.
IMO, Python's extensive introspection features make that a losing
battle, since it's simply too easy to forget to blacklist
something and the attacker finding it. Not only that, even with a
perfect vacuum-sealed jail, an attacker can still bring down
the interpreter by exhausting memory or consuming excess CPU.

I think I might have a way of securely sealing-in untrusted code. It's a
fairly nascent idea, though, and I haven't worked out
all of the details yet, so I'm posting what I have so far for feedback
and for others to try to poke holes in it.

Absolutely nothing here is final. I'm just framing out what I generally
had in mind. Obviously, it will need to be adjusted to
be consistent with "the Python way" - my hope is that this can become a
PEP. :)


>>> # It all starts with the introduction of a new type, called a jail.
(I haven't yet worked out whether it should be a builtin type,
... # or a module.) Unjailed code can create jails, which will run the
untrusted code and keep strict limits on it.
...
>>> j = jail()
>>> dir(j)
['__class__', '__delattr__', '__doc__', '__format__',
'__getattribute__', '__hash__',
'__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setattr__',
'__sizeof__', '__str__', '__subclasshook__', 'acquire', 'getcpulimit',
'getcpuusage',
'getmemorylimit', 'getmemoryusage', 'gettimelimit', 'gettimeusage',
'release',
'setcpulimit', 'setmemorylimit', 'settimelimit']
>>> # The jail monitors three things: Memory (in bytes), real time (in
seconds), and CPU time (also in seconds)
... # and it also allows you to impose limits on them. If any limit is
non-zero, code in that jail may not exceed its limit.
... # Exceeding a memory limit will result in a MemoryError. I haven't
decided what CPU/real time limits should raise.
... # The other two calls are "acquire" and "release," which allow you
to seal (any) objects inside the jail, or bust them
    # out. Objects inside the jail (i.e. created by code in that jail)
contribute their __sizeof__() to the j.getmemoryusage()
...
>>> def stealPasswd():
...         return open('/etc/passwd','r').read()
...
>>> j.acquire(stealPasswd)
>>> j.getmemoryusage() # The stealPasswd function, its code, etc. are
now locked away within the jail.
375
>>> stealPasswd()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
JailError: tried to access an object outside of the jail

The object in question is, of course, 'open'. Unlike the f_restricted
model, the jail was freely able to grab
the open() function, but was absolutely unable to touch it: It can't
call it, set/get/delete attributes/items,
or pass it as an argument to any functions. There are three criteria
that determine whether an object can
be accessed:
a. The code accessing the object is not within a jail; or
b. The object belongs to the same jail as the code accessing the object; or
c. The object has an __access__ function, and
theObject.__access__(theJail) returns True.

For the jail to be able to access 'open', it needs to be given access
explicitly. I haven't quite decided
how this should work, but I had in mind the creation of a "guard"
(essentially a proxy) that allows the jail
to access the object. It belongs to the same jail as the guarded object
(and is therefore impossible to create
within a jail unless the guarded object belongs to the same jail), has a
list of jails (or None for 'any') that the
guard will allow to __access__ it (the guard is immutable, so jails
can't mess with it even though they can
access it), and what the guard will allow though it (read-write,
read-only, call-within-jail, call-outside-jail).

I have a couple remaining issues that I haven't quite sussed out:
* How exactly do guards work? I had in mind a system of proxies (memory
usage is a concern, especially
    in memory-limited jails - maybe allow __access__ to return specific
modes of access rather than
    all-or-nothing?) that recursively return more guards after
operations. (e.g., if I have a guard allowing
    read+call on sys, sys.stdout would return another guard allowing
read+call on sys.stdout, likewise for
    sys.stdout.write)
* How are objects sealed in the jail? j.acquire can lead to serious
problems with lots of references
    getting recursively sealed in. Maybe disallow sealing in anything
but code objects, or allow explicitly
    running code within a jail like j.execute(code, globals(),
locals()), which works fine since any objects
    created by jailed code are also jailed.
* How do imports work? Should __import__ be modified so that when a jail
invokes it, the import runs
    normally (unjailed), and then returns the module with a special
guard that allows read-only+call-within,
    but not on builtins? This has a nice advantage, since jailed code
can import e.g. socket, and maybe even
    create a socket, but won't be able to do sock.connect(...), since
socket.connect (which is running with
    jailed permissions) can't touch the builtin _socket module.
* Is obj.__access__(j) the best way to decide access? It doesn't allow
programmers much freedom to
    customize the jail policy since they can't modify __access__ for
builtins. Maybe the jail should have
    the first chance (such as j.__access__(obj)), which allows
programmers to subclass the jail, and the jail
    can fallback to obj.__access__(j)
* How does Python keep track of what jail each frame is in? Maybe each
frame can have a frame.f_jail,
    which references the jail object restricting that frame (or None for
unjailed code) - frames' jails default
    to the jail holding the code object, or can be explicitly overridden
(as in j.execute(code, globals(), locals()))
* When are jails switched? Obviously, jailed code called from unjailed
code (or even from other unjailed
    code) should be executed in the callee jail... But if a jailed
caller is calling unjailed code, does the jail
    follow, or does the unjailed code run in an unjailed frame? How do
programmers specify that?

...that's pretty much my two (erm, twenty) cents on the matter. Again,
any feedback/adversarial reasoning
you guys can offer is much appreciated.

From rdmurray at bitdance.com  Sat Jun 11 02:41:32 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 10 Jun 2011 20:41:32 -0400
Subject: [Python-Dev] Python jails
In-Reply-To: <4DF2B593.8040302@colorado.edu>
References: <4DF2B593.8040302@colorado.edu>
Message-ID: <20110611004134.0ABB62505A0@webabinitio.net>

On Fri, 10 Jun 2011 18:23:47 -0600, Sam Edwards <sam.edwards at Colorado.EDU> wrote:
> Hello! This is my first posting to the python-dev list, so please
> forgive me if I violate any unspoken etiquette here. :)

Well, hopefully we won't bite, though of course I can't promise anything
for anyone else :)

I haven't read through your post, but if you don't know about it I
suspect that you will be interested in the following:

    http://code.activestate.com/pypm/pysandbox/

I'm pretty sure Victor will be happy to have someone else interested in
this topic.

--
R. David Murray           http://www.bitdance.com

From guido at python.org  Sat Jun 11 02:44:26 2011
From: guido at python.org (Guido van Rossum)
Date: Fri, 10 Jun 2011 17:44:26 -0700
Subject: [Python-Dev] Python jails
In-Reply-To: <4DF2B593.8040302@colorado.edu>
References: <4DF2B593.8040302@colorado.edu>
Message-ID: <BANLkTik+c3BWSHPHEy5=8nemPrL7xnnPRg@mail.gmail.com>

Hi Sam,

Have you seen this?
http://tav.espians.com/paving-the-way-to-securing-the-python-interpreter.html

It might relate a similar idea. There were a few iterations of Tav's approach.

--Guido

On Fri, Jun 10, 2011 at 5:23 PM, Sam Edwards <sam.edwards at colorado.edu> wrote:
> Hello! This is my first posting to the python-dev list, so please
> forgive me if I violate any unspoken etiquette here. :)
>
> I was looking at Python 2.x's f_restricted frame flag (or, rather, the
> numerous ways around it) and noticed that most (all?)
> of the attacks to escape restricted execution involved the attacker
> grabbing something he wasn't supposed to have.
> IMO, Python's extensive introspection features make that a losing
> battle, since it's simply too easy to forget to blacklist
> something and the attacker finding it. Not only that, even with a
> perfect vacuum-sealed jail, an attacker can still bring down
> the interpreter by exhausting memory or consuming excess CPU.
>
> I think I might have a way of securely sealing-in untrusted code. It's a
> fairly nascent idea, though, and I haven't worked out
> all of the details yet, so I'm posting what I have so far for feedback
> and for others to try to poke holes in it.
>
> Absolutely nothing here is final. I'm just framing out what I generally
> had in mind. Obviously, it will need to be adjusted to
> be consistent with "the Python way" - my hope is that this can become a
> PEP. :)
>
>
>>>> # It all starts with the introduction of a new type, called a jail.
> (I haven't yet worked out whether it should be a builtin type,
> ... # or a module.) Unjailed code can create jails, which will run the
> untrusted code and keep strict limits on it.
> ...
>>>> j = jail()
>>>> dir(j)
> ['__class__', '__delattr__', '__doc__', '__format__',
> '__getattribute__', '__hash__',
> '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
> '__setattr__',
> '__sizeof__', '__str__', '__subclasshook__', 'acquire', 'getcpulimit',
> 'getcpuusage',
> 'getmemorylimit', 'getmemoryusage', 'gettimelimit', 'gettimeusage',
> 'release',
> 'setcpulimit', 'setmemorylimit', 'settimelimit']
>>>> # The jail monitors three things: Memory (in bytes), real time (in
> seconds), and CPU time (also in seconds)
> ... # and it also allows you to impose limits on them. If any limit is
> non-zero, code in that jail may not exceed its limit.
> ... # Exceeding a memory limit will result in a MemoryError. I haven't
> decided what CPU/real time limits should raise.
> ... # The other two calls are "acquire" and "release," which allow you
> to seal (any) objects inside the jail, or bust them
> ? ?# out. Objects inside the jail (i.e. created by code in that jail)
> contribute their __sizeof__() to the j.getmemoryusage()
> ...
>>>> def stealPasswd():
> ... ? ? ? ? return open('/etc/passwd','r').read()
> ...
>>>> j.acquire(stealPasswd)
>>>> j.getmemoryusage() # The stealPasswd function, its code, etc. are
> now locked away within the jail.
> 375
>>>> stealPasswd()
> Traceback (most recent call last):
> ?File "<stdin>", line 1, in <module>
> JailError: tried to access an object outside of the jail
>
> The object in question is, of course, 'open'. Unlike the f_restricted
> model, the jail was freely able to grab
> the open() function, but was absolutely unable to touch it: It can't
> call it, set/get/delete attributes/items,
> or pass it as an argument to any functions. There are three criteria
> that determine whether an object can
> be accessed:
> a. The code accessing the object is not within a jail; or
> b. The object belongs to the same jail as the code accessing the object; or
> c. The object has an __access__ function, and
> theObject.__access__(theJail) returns True.
>
> For the jail to be able to access 'open', it needs to be given access
> explicitly. I haven't quite decided
> how this should work, but I had in mind the creation of a "guard"
> (essentially a proxy) that allows the jail
> to access the object. It belongs to the same jail as the guarded object
> (and is therefore impossible to create
> within a jail unless the guarded object belongs to the same jail), has a
> list of jails (or None for 'any') that the
> guard will allow to __access__ it (the guard is immutable, so jails
> can't mess with it even though they can
> access it), and what the guard will allow though it (read-write,
> read-only, call-within-jail, call-outside-jail).
>
> I have a couple remaining issues that I haven't quite sussed out:
> * How exactly do guards work? I had in mind a system of proxies (memory
> usage is a concern, especially
> ? ?in memory-limited jails - maybe allow __access__ to return specific
> modes of access rather than
> ? ?all-or-nothing?) that recursively return more guards after
> operations. (e.g., if I have a guard allowing
> ? ?read+call on sys, sys.stdout would return another guard allowing
> read+call on sys.stdout, likewise for
> ? ?sys.stdout.write)
> * How are objects sealed in the jail? j.acquire can lead to serious
> problems with lots of references
> ? ?getting recursively sealed in. Maybe disallow sealing in anything
> but code objects, or allow explicitly
> ? ?running code within a jail like j.execute(code, globals(),
> locals()), which works fine since any objects
> ? ?created by jailed code are also jailed.
> * How do imports work? Should __import__ be modified so that when a jail
> invokes it, the import runs
> ? ?normally (unjailed), and then returns the module with a special
> guard that allows read-only+call-within,
> ? ?but not on builtins? This has a nice advantage, since jailed code
> can import e.g. socket, and maybe even
> ? ?create a socket, but won't be able to do sock.connect(...), since
> socket.connect (which is running with
> ? ?jailed permissions) can't touch the builtin _socket module.
> * Is obj.__access__(j) the best way to decide access? It doesn't allow
> programmers much freedom to
> ? ?customize the jail policy since they can't modify __access__ for
> builtins. Maybe the jail should have
> ? ?the first chance (such as j.__access__(obj)), which allows
> programmers to subclass the jail, and the jail
> ? ?can fallback to obj.__access__(j)
> * How does Python keep track of what jail each frame is in? Maybe each
> frame can have a frame.f_jail,
> ? ?which references the jail object restricting that frame (or None for
> unjailed code) - frames' jails default
> ? ?to the jail holding the code object, or can be explicitly overridden
> (as in j.execute(code, globals(), locals()))
> * When are jails switched? Obviously, jailed code called from unjailed
> code (or even from other unjailed
> ? ?code) should be executed in the callee jail... But if a jailed
> caller is calling unjailed code, does the jail
> ? ?follow, or does the unjailed code run in an unjailed frame? How do
> programmers specify that?
>
> ...that's pretty much my two (erm, twenty) cents on the matter. Again,
> any feedback/adversarial reasoning
> you guys can offer is much appreciated.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From pje at telecommunity.com  Sat Jun 11 02:52:28 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 10 Jun 2011 20:52:28 -0400
Subject: [Python-Dev] Python jails
In-Reply-To: <4DF2B593.8040302@colorado.edu>
References: <4DF2B593.8040302@colorado.edu>
Message-ID: <20110611005243.737573A4060@sparrow.telecommunity.com>

At 06:23 PM 6/10/2011 -0600, Sam Edwards wrote:
>I have a couple remaining issues that I haven't quite sussed out:
>[long list of questions deleted]

You might be able to answer some of them by looking at this project:

   http://pypi.python.org/pypi/RestrictedPython

Which implements the necessary ground machinery for doing that sort 
of thing, in the form of a specialized Python compiler (implemented 
in Python, for 2.3 through 2.7) that allows you to implement whatever 
sorts of guards and security policies you want on top of it.

Even if it doesn't answer all your questions in and of itself, it may 
prove a fruitful environment in which you can experiment with various 
approaches and see which ones you actually like, without first having 
to write a bunch of code yourself.

Discussing an official implementation of this sort of thing as a 
language feature is probably best left to python-ideas, though, until 
and unless you actually have a PEP to propose.


From sam.edwards at Colorado.EDU  Sat Jun 11 06:08:55 2011
From: sam.edwards at Colorado.EDU (Sam Edwards)
Date: Fri, 10 Jun 2011 22:08:55 -0600
Subject: [Python-Dev] Python jails
In-Reply-To: <4DF2B593.8040302@colorado.edu>
References: <4DF2B593.8040302@colorado.edu>
Message-ID: <4DF2EA57.3030506@colorado.edu>

All,

Thanks for the quick responses!

I've skimmed the pysandbox code yesterday. I think Victor has the right
idea with relying on a whitelist, as well as limiting execution time.
The fact that untrusted code can still execute memory exhaustion attacks
is the only thing that still worries me: It's hard to write a server
that will run hundreds of scripts from untrusted users, since one of
them can bring down the entire server by writing an infinite loop that
allocates tons of objects. Python needs a way to hook the
object-allocation process in order to (effectively) limit how much
memory untrusted code can consume.

Tav's blog post makes some interesting points... The object-capability
model definitely has the benefit of efficiency; simply getting the
reference to an object means the untrusted code is trusted with full
capability to that object (which saves having to query the jail every
time the object is touched) - it's just as fast as unrestricted Python,
which I like. Perhaps my jails idea should then be refactored into some
mechanism for monitoring and limiting memory and CPU usage -- it's the
perfect thing to ship as an extension, the only shame is that it
requires interpreter support.
Anyway, in light of Tav's post which seems to suggest that f_restricted
frames are impossible to escape (if used correctly), why was
f_restricted removed in Python 3? Is it simply that it's too easy to
make a mistake and accidentally give an attacker an unsafe object, or is
there some fundamental flaw with it? Could you see something like
f_restricted (or f_jail) getting put back in Python 3, if it were a good
deal more bulletproof?

And, yeah, I've been playing with RestrictedPython. It's pretty good,
but it lacks memory- and CPU-limiting, which is my main focus right now.
And yes, I should probably have posted this to python-ideas, thanks. :)
This is a very long way away from a PEP.

PyPy's sandboxing feature is probably closest to what I'd like, but I'm
looking for something that can coexist in the same process (since
running hundreds of interpreter processes continuously has a lot of
system memory overhead, it's better if the many untrusted, but
independent, jails could share a single interpreter)

From victor.stinner at haypocalc.com  Sat Jun 11 10:34:30 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 11 Jun 2011 10:34:30 +0200
Subject: [Python-Dev] Python jails
In-Reply-To: <20110611004134.0ABB62505A0@webabinitio.net>
References: <4DF2B593.8040302@colorado.edu>
	<20110611004134.0ABB62505A0@webabinitio.net>
Message-ID: <4DF32896.8020504@haypocalc.com>

Le 11/06/2011 02:41, R. David Murray a ?crit :
> I haven't read through your post, but if you don't know about it I
> suspect that you will be interested in the following:
>
>      http://code.activestate.com/pypm/pysandbox/
>
> I'm pretty sure Victor will be happy to have someone else interested in
> this topic.
>    
Yes, I am happy :-) The project URL is https://github.com/haypo/pysandbox/

Activestate page is wrong: pysanbox does support Python 3 (Python 2.5 - 
3.3).

pysandbox uses different policy depending on the problem. For example, 
whitelist for builtins, blacklist for object attributes. pysandbox is 
based on Tav's ideas.

The main idea of pysandbox is to execute untrusted in a new empty 
namespace, the untrusted namespace. Objects imported into this namespace 
are imported as proxies to get a read-only view of the Python namespace. 
Importing modules is protected by a whitelist (modules and symbols 
names). To protect the namespace, some introspection attributes are 
hidden like __subclasses__ or __self__. Performances are supposed to be 
close to a classic Python interpreter (I didn't run a benchmark, I don't 
really care).

An empty namespace is not enough to protect Python: pysandbox denies the 
execution of arbitrary bytecode, write files, write to stdout/stderr, 
exit Python, etc. Tav's sandbox is good to deny everything, whereas you 
can configure pysandbox to enable some features (e.g. exit Python, 
useful for an interpreter).

About restricted mode: you can also configure pysandbox to use it, but 
the restricted mode is too much restrictive: you cannot open files, 
whereas pysandbox allows to read files in a whitelist (e.g. useful to 
display a backtrace).

If you would like to implement your own sandbox: great! You should try 
pysandbox test suite, I'm proud of it :-)

I am still not sure that pysandbox approach is the good one: if you find 
a vulnerability to escape pysandbox "jail" (see pysandbox Changelog, it 
would not be the first time), you can do anything. PyPy sandbox and 
"Seccomp nurse" (specific to Linux?) use two processes: the Python 
process cannot do anything, it relies completly in a trusted process 
which control all operations. I don't understand exactly how it is 
different: a vulnerability in the trusted process gives also full 
control, but it's maybe a safer approach. Or the difference is maybe 
that the implementation is simpler (less code?) and so safer (less code 
usually means less bugs).

"Seccomp nurse":
http://chdir.org/~nico/seccomp-nurse/

I tested recently AppEngine sandbox (testable online via 
http://shell.appspot.com/): it is secure *and* powerful, quite all 
modules are allowed (except not ctypes, as expected). AppEngine is not a 
Python sandbox: it's a sandbox between the Python process and Linux 
kernel, so it protects also modules written in C (pysandbox is unable to 
protect these modules). AppEngine modifies the Python standard library 
to cooperate with the low-level sandbox, e.g. raise nice error messages 
with open(filename, "w"): invalid file mode (instead of an ugly OSError 
with a cryptic message).

Get more information about pysandbox and other sandboxes in pysandbox 
README file.

Victor

From ncoghlan at gmail.com  Sat Jun 11 11:16:37 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 11 Jun 2011 19:16:37 +1000
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
In-Reply-To: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
References: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
Message-ID: <BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>

On Sat, Jun 11, 2011 at 7:15 AM, Ben Wolfson <wolfson at gmail.com> wrote:
[snip very thorough analysis]

To summarise (after both the above post and the discussion on the tracker)

The current str.format implementation differs from the documentation
in two ways:

1. It ignores the presence of an unclosed index field when processing
a replacement field (placing additional restrictions on allowable
characters in index strings).
2. Replacement fields that appear in name specifiers are processed by
the parser for brace-matching purposes, but not substituted

More accurate documentation would state that:

1. Numeric name fields start with a digit and are terminated by any
non-numeric character.

2. An identifier name field is terminated by any one of:
    '}' (terminates the replacement field, unless preceded by a
matching '{' character, in which case it is ignored and included in
the string)
    '!' (terminates name field, starts conversion specifier)
    ':' (terminates name field, starts format specifier)
    '.' (terminates current name field, starts new name field for subattribute)
    '[' (terminates name field, starts index field)

3. An index field is terminated by one of:
    '}' (terminates the replacement field, unless preceded by a
matching '{' character, in which case it is ignored and included in
the string)
    '!' (terminates index field, starts conversion specifier)
    ':' (terminates index field, starts format specifier)
    ']' (terminates index field, subsequent character will determine next field)

This existing behaviour can certainly be documented as such, but is
rather unintuitive and (given that '}', '!' and ']' will always error
out if appearing in an index field) somewhat silly.

So, the two changes that I believe Ben is proposing would be as follows:

1. When processing a name field, brace-matching is suspended. Between
the opening '{' character and the closing '}', '!' or ':' character,
additional '{' characters are ignored for matching purposes.
2. When processing an index field, all special processing is suspended
until the terminating ']' is reached

The rules for name fields would then become:

1. Numeric fields start with a digit and are terminated by any
non-numeric character.

2. An identifier name field is terminated by any one of:
    '}' (terminates the replacement field)
    '!' (terminates identifier field, starts conversion specifier)
    ':' (terminates identifier field, starts format specifier)
    '.' (terminates identifier field, starts new identifier field for
subattribute)
    '[' (terminates identifier field, starts index field)

3. An index field is terminated by ']' (subsequent character will
determine next field)

That second set of rules is *far* more in line with the behaviour of
the rest of the language than the status quo, so unless the difficulty
of making the str.format mini-language parser work that way is truly
prohibitive, it certainly seems worthwhile to tidy up the semantics.

The index field behaviour should definitely be fixed, as it poses no
backwards compatibility concerns. The brace matching behaviour should
probably be left alone, as changing it would potentially break
currently valid format strings (e.g. "{a{0}}".format(**{'a{0}':1})
produces '1' now, but would raise an exception if the brace matching
rules were changed).

So +1 on making the str.format parser accept anything other than ']'
inside an index field and turn the whole thing into an ordinary
string, -1 on making any other changes to the brace-matching
behaviour.

That would leave us with the following set of rules for name fields:

1. Numeric fields start with a digit and are terminated by any
non-numeric character.

2. An identifier name field is terminated by any one of:
    '}' (terminates the replacement field, unless preceded by a
matching '{' character, in which case it is ignored and included in
the string)
    '!' (terminates identifier field, starts conversion specifier)
    ':' (terminates identifier field, starts format specifier)
    '.' (terminates identifier field, starts new identifier field for
subattribute)
    '[' (terminates identifier field, starts index field)

3. An index field is terminated by ']' (subsequent character will
determine next field)

Note that brace-escaping currently doesn't work inside name fields, so
that should also be fixed:

>>> "{0[{{]}".format({'{':1})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmatched '{' in format
>>> "{a{{}".format(**{'a{':1})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: unmatched '{' in format

As far as I can recall, the details of this question didn't come up
when PEP 3101 was developed, so the PEP isn't a particularly good
source to justify anything in relation to this - it is best to
consider the current behaviour to just be the way it happened to be
implemented rather than a deliberate design choice.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From petri at digip.org  Sat Jun 11 12:32:13 2011
From: petri at digip.org (Petri Lehtinen)
Date: Sat, 11 Jun 2011 13:32:13 +0300
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
In-Reply-To: <BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>
References: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
	<BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>
Message-ID: <20110611103133.GA27053@toronto>

Nick Coghlan wrote:
[snip]
> The rules for name fields would then become:
> 
> 1. Numeric fields start with a digit and are terminated by any
> non-numeric character.
> 
> 2. An identifier name field is terminated by any one of:
>     '}' (terminates the replacement field)
>     '!' (terminates identifier field, starts conversion specifier)
>     ':' (terminates identifier field, starts format specifier)
>     '.' (terminates identifier field, starts new identifier field for
> subattribute)
>     '[' (terminates identifier field, starts index field)
> 
> 3. An index field is terminated by ']' (subsequent character will
> determine next field)

+1

> That second set of rules is *far* more in line with the behaviour of
> the rest of the language than the status quo, so unless the difficulty
> of making the str.format mini-language parser work that way is truly
> prohibitive, it certainly seems worthwhile to tidy up the semantics.
> 
> The index field behaviour should definitely be fixed, as it poses no
> backwards compatibility concerns. The brace matching behaviour should
> probably be left alone, as changing it would potentially break
> currently valid format strings (e.g. "{a{0}}".format(**{'a{0}':1})
> produces '1' now, but would raise an exception if the brace matching
> rules were changed).

-1 for leaving the brace matching behavior alone, as it's very
unintuitive for *the user*. For the implementor it may make sense to
count matching braces, but definitely not for the user. I don't
believe that "{a{0}}" is a real use case that someone might already
use, as it's a hard violation of what the documentation currently
says.

I'd rather disallow braces in the replacement field before the format
specifier altogether. Or closing braces at the minimum. Furthermore,
the double-escaping sounds reasonable in the format specifier, but not
elsewhere.

My motivation is that the user should be able to have a quick glance
on the format string and see where the replacement fields are. This is
probably what the PEP intends to say when disallowing braces inside
the replacement field. In my opinion, it's easy to write the parser in
a way that braces are parsed in any imaginable manner. Or maybe not
easy, but not harder than any other way of handling braces.

> So +1 on making the str.format parser accept anything other than ']'
> inside an index field and turn the whole thing into an ordinary
> string, -1 on making any other changes to the brace-matching
> behaviour.
> 
> That would leave us with the following set of rules for name fields:
> 
> 1. Numeric fields start with a digit and are terminated by any
> non-numeric character.
> 
> 2. An identifier name field is terminated by any one of:
>     '}' (terminates the replacement field, unless preceded by a
> matching '{' character, in which case it is ignored and included in
> the string)
>     '!' (terminates identifier field, starts conversion specifier)
>     ':' (terminates identifier field, starts format specifier)
>     '.' (terminates identifier field, starts new identifier field for
> subattribute)
>     '[' (terminates identifier field, starts index field)
> 
> 3. An index field is terminated by ']' (subsequent character will
> determine next field)
> 
> Note that brace-escaping currently doesn't work inside name fields, so
> that should also be fixed:
> 
> >>> "{0[{{]}".format({'{':1})
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: unmatched '{' in format
> >>> "{a{{}".format(**{'a{':1})
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: unmatched '{' in format

-1. Why do we need braces inside replacement fields at all (except for
inner replacements in the format specier)? I strongly believe that the
PEP's use case is the simple one:

    '{foo}'.format(foo=10)

In my opinoin, these '{!#%}'.format(**{'!#%': 10}) cases are not real.
The current documentation requires field_name to be a valid
identifier, an this is a sane requirement. The only problem is that
parsing identifiers correctly is very hard, so it can be made simpler
by allowing some non-identifiers. But we still don't have to accept
braces.

---

As a somewhat another issue, I'm confused about this:

  >>> '{a[1][2]}'.format(a={1:{2:3}})
  '3'

and even more about this:

  >>> '{a[1].foo[2]}'.format(a={1:namedtuple('x', 'foo')({2:3})})
  '3'

Why does this work? It's against the current documentation. The
documented syntax only allows zero or one attribute names and zero or
one element index, in this order. Is it intentional that we allow
arbitrary chains of getattr and __getitem__? If we do, this should be
documented, too.

Petri

From wolfson at gmail.com  Sat Jun 11 18:19:50 2011
From: wolfson at gmail.com (Ben Wolfson)
Date: Sat, 11 Jun 2011 09:19:50 -0700
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
In-Reply-To: <BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>
References: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
	<BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>
Message-ID: <BANLkTino_LOtR=UcTT4UfV=kGtUiaKNG_A@mail.gmail.com>

On Sat, Jun 11, 2011 at 2:16 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sat, Jun 11, 2011 at 7:15 AM, Ben Wolfson <wolfson at gmail.com> wrote:
> To summarise (after both the above post and the discussion on the tracker)

Thanks for the summary!

>
> That would leave us with the following set of rules for name fields:
>
> 1. Numeric fields start with a digit and are terminated by any
> non-numeric character.
>
> 2. An identifier name field is terminated by any one of:
> ? ?'}' (terminates the replacement field, unless preceded by a
> matching '{' character, in which case it is ignored and included in
> the string)
> ? ?'!' (terminates identifier field, starts conversion specifier)
> ? ?':' (terminates identifier field, starts format specifier)
> ? ?'.' (terminates identifier field, starts new identifier field for
> subattribute)
> ? ?'[' (terminates identifier field, starts index field)
>
> 3. An index field is terminated by ']' (subsequent character will
> determine next field)

A minor clarification since I mentioned a patch: the patch as it
exists implements *these*---Nick's---semantics. That is, it will allow
these:

"{0.{a}}".format(x)
"{0.{[{].}}".format(x)

But not this, because it keeps current brace-matching in this context:

"{0.{a}".format(x)

And it treats this:

"{0.a}}".format(x)

as the markup "{0.a}" followed by the character data "}".

The patch would have to be changed to turn off brace balancing in name
fields as well.

In either case there would be potential breakage, since this:

"{0[{}.}}".format(...)

currently works, but would not work anymore, under either set of
rules. (The likelihood that this potential breakage would anywhere be
actual breakage is pretty slim, though.)

> Note that brace-escaping currently doesn't work inside name fields, so
> that should also be fixed:
>
>>>> "{0[{{]}".format({'{':1})
> Traceback (most recent call last):
> ?File "<stdin>", line 1, in <module>
> ValueError: unmatched '{' in format
>>>> "{a{{}".format(**{'a{':1})
> Traceback (most recent call last):
> ?File "<stdin>", line 1, in <module>
> ValueError: unmatched '{' in format

This is a slightly different issue, though, isn't it? As far as I can
tell, if the brace-matching rules are kept in place, there would never
be any *need* for escaping. You can't have an internal replacement
field in this part of the replacement field, so '{' can always safely
be assumed to be Just a Brace and not the start of a replacement
field, regardless of whether it's doubled, and '}' will either be in
an index field (where it can't have the significance of ending the
replacement field) or it will be (a) the end of the replacement field
or (b) not the end of the replacement field because matched by an
earlier '{'. So there would never be any role for escaping to play.

There would be a role for escaping if the rules for name fields are
that '}' terminates them, no matching done; then, you could double
them to get a '}' in the name field. But, to be honest, that strikes
me as introducing a lot of heavy machinery for very little gain;
opening and closing braces would have to be escaped to accomodate this
one thing. And it's not as if you can escape ']' in index
fields---which would be a parallel case. It seems significantly
simpler to me to leave the escaping behavior as it is in this part of
the replacement field.

-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]

From benjamin at python.org  Sat Jun 11 20:28:16 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 11 Jun 2011 13:28:16 -0500
Subject: [Python-Dev] [Python-checkins] cpython (3.1): onto 3.1.5
In-Reply-To: <4DF3B324.1010708@udel.edu>
References: <E1QVR8R-0005ws-6U@dinsdale.python.org> <4DF3B324.1010708@udel.edu>
Message-ID: <BANLkTi=kPxefrb5vpT_PcxrksP9HRBrWyQ@mail.gmail.com>

2011/6/11 Terry Reedy <tjreedy at udel.edu>:
>
>> +What's New in Python 3.1.5?
>> +===========================
>> +
>> +*Release date: XXXX-XX-XX*
>> +
>> +Core and Builtins
>> +-----------------
>> +
>> +Library
>> +-------
>> +
>> +
>
> I presume that only security patches should be added.

Indeed.


-- 
Regards,
Benjamin

From tjreedy at udel.edu  Sat Jun 11 21:17:49 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 11 Jun 2011 15:17:49 -0400
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
In-Reply-To: <20110611103133.GA27053@toronto>
References: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>	<BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>
	<20110611103133.GA27053@toronto>
Message-ID: <it0f0v$jer$1@dough.gmane.org>

On 6/11/2011 6:32 AM, Petri Lehtinen wrote:
> Nick Coghlan wrote:
> [snip]

It seems to me that the intent of the pep and the current doc is that 
field_names should match what one would write in code except that quotes 
are left off of literal string keys. Which is to say, the brackets [] 
serve as quote marks. So once '[' is found, the scanner must shift to 
'in index' mode and accept everything until a matching ']' is found, 
ending 'in index' mode.

The arg_name is documented as int or identifier and attribute_name as 
identifier, period. Anything more than that is an implementation 
accident which people should not count on in either future versions or 
alternate implementations.

I can imagine uses for nested replacement fields in the field_name or 
conversion spec. Ie, '{ {0}:{1}d'.format(2,5,333,444) == '  333', 
whereas changing the first arg to 3 would produce '  444'. If braces are 
allowed in either of the first two segments (outside of the 'quoted' 
within braces context), I think it should only be for the purpose of a 
feature addition that makes them significant.

It strikes me that the underlying problem is that the replacement_field 
scanner is, apparently, hand-crafted rather than being generated from 
the corresponding grammar, as is the overall Python lexer-parser. So it 
has no necessary connection with the grammar.

-- 
Terry Jan Reedy


From greg.ewing at canterbury.ac.nz  Sun Jun 12 01:29:23 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 12 Jun 2011 11:29:23 +1200
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
In-Reply-To: <BANLkTino_LOtR=UcTT4UfV=kGtUiaKNG_A@mail.gmail.com>
References: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
	<BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>
	<BANLkTino_LOtR=UcTT4UfV=kGtUiaKNG_A@mail.gmail.com>
Message-ID: <4DF3FA53.7000906@canterbury.ac.nz>

Ben Wolfson wrote:
> You can't have an internal replacement
> field in this part of the replacement field, so '{' can always safely
> be assumed to be Just a Brace and not the start of a replacement
> field, regardless of whether it's doubled,

I'm worried that the rules in this area are getting too
complicated for a human to follow. If braces are allowed
as plain data between square brackets and/or vice versa,
it's going to be a confusing mess to read, and there will
always be some doubt in the programmer's mind as to whether
they have to be escaped somehow or not.

I'm inclined to think that any such difficult cases should
simply be disallowed. If the docs say an identifier is required
someplace, the implementation should adhere strictly to that.

It's not *that* hard to parse an indentifier properly, and
IMO any use case that requires putting arbitrary characters
into an item selector is abusing the format mechanism and
should be redesigned to work some other way.

-- 
Greg

From ncoghlan at gmail.com  Sun Jun 12 15:18:37 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Jun 2011 23:18:37 +1000
Subject: [Python-Dev] [Python-checkins] cpython (2.7): allow "fake"
 filenames in findsource (closes #9284)
In-Reply-To: <E1QVVE7-0006sB-Fh@dinsdale.python.org>
References: <E1QVVE7-0006sB-Fh@dinsdale.python.org>
Message-ID: <BANLkTi=XxvC7Vu-+paNe03gXXVVfzCwhfA@mail.gmail.com>

On Sun, Jun 12, 2011 at 6:56 AM, benjamin.peterson
<python-checkins at python.org> wrote:
> summary:
> ?allow "fake" filenames in findsource (closes #9284)
>
> This allows findsource() to work in doctests.
>
> A patch from Dirkjan Ochtman.

Either this exception should be mentioned in the inspect.getsource()
documentation or else doctest should be monkeypatching inspect as well
as linecache.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From benjamin at python.org  Sun Jun 12 17:31:11 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 12 Jun 2011 10:31:11 -0500
Subject: [Python-Dev] [Python-checkins] cpython (2.7): allow "fake"
 filenames in findsource (closes #9284)
In-Reply-To: <BANLkTi=XxvC7Vu-+paNe03gXXVVfzCwhfA@mail.gmail.com>
References: <E1QVVE7-0006sB-Fh@dinsdale.python.org>
	<BANLkTi=XxvC7Vu-+paNe03gXXVVfzCwhfA@mail.gmail.com>
Message-ID: <BANLkTikyZ3=6yAipy7Gyu1cLq3tsxzqbjw@mail.gmail.com>

2011/6/12 Nick Coghlan <ncoghlan at gmail.com>:
> On Sun, Jun 12, 2011 at 6:56 AM, benjamin.peterson
> <python-checkins at python.org> wrote:
>> summary:
>> ?allow "fake" filenames in findsource (closes #9284)
>>
>> This allows findsource() to work in doctests.
>>
>> A patch from Dirkjan Ochtman.
>
> Either this exception should be mentioned in the inspect.getsource()
> documentation or else doctest should be monkeypatching inspect as well
> as linecache.

I should have made more clear in the message that this is actually a
regression from 2.6.



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Sun Jun 12 17:55:13 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2011 01:55:13 +1000
Subject: [Python-Dev] [Python-checkins] cpython (2.7): allow "fake"
 filenames in findsource (closes #9284)
In-Reply-To: <BANLkTikyZ3=6yAipy7Gyu1cLq3tsxzqbjw@mail.gmail.com>
References: <E1QVVE7-0006sB-Fh@dinsdale.python.org>
	<BANLkTi=XxvC7Vu-+paNe03gXXVVfzCwhfA@mail.gmail.com>
	<BANLkTikyZ3=6yAipy7Gyu1cLq3tsxzqbjw@mail.gmail.com>
Message-ID: <BANLkTikcdFeA2GKpHky1DPreMiWoq0TWvw@mail.gmail.com>

On Mon, Jun 13, 2011 at 1:31 AM, Benjamin Peterson <benjamin at python.org> wrote:
> I should have made more clear in the message that this is actually a
> regression from 2.6.

Actually looking at the inspect docs, I'm not sure where such a note
would fit anyway. I'll think about it a bit more - I have a suspicion
there may be a flawed assumption in that inspect code and it should be
passing more queries through to linecache rather than trying to
second-guess it regarding what source code is available.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From jiawei.h.li at gmail.com  Sat Jun 11 10:29:16 2011
From: jiawei.h.li at gmail.com (Jiawei Li)
Date: Sat, 11 Jun 2011 16:29:16 +0800
Subject: [Python-Dev] Implement Aspect-oriented programming
Message-ID: <BANLkTin46+QbM3ScxC=vw0z-GpU_ZqcvDw@mail.gmail.com>

For example, the logging module is not very useful right now, as it requires
sprinkling small one-liners all over my code - not exactly ideal.
Why not take a page from aspect-oriented programming and allow for injection
of code with point cuts?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110611/197aa4f9/attachment.html>

From lukas.lueg at googlemail.com  Sun Jun 12 17:29:13 2011
From: lukas.lueg at googlemail.com (Lukas Lueg)
Date: Sun, 12 Jun 2011 17:29:13 +0200
Subject: [Python-Dev] Lazy unpacking for struct module
Message-ID: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>

Hi.

We extensively use the struct module to crunch large amounts of binary
data. There are basically two operations for us that only seem to the
naked eye as one: Filtering (see if certain fields have certain
values, throw everything away if not) and inspection (care about all
the fields' values). The filtering-part is very important as most of
the binary data can actually be thrown away and never have to be
inspected any further. When thinking about how to increase
performance, one thought was that a lot of objects are generated by
the struct module that we never need: Unpacking six fields in order to
look at one and then throwing everything away is inefficient
concerning the five other fields. It also makes filtering and
inspecting basically the same operation regarding the (slow) unpacking
of values so we don't really benefit from filtering. This is a huge
problem when crunching gigabytes of data and creating millions of
fields.

One solution to this is using two format-strings instead of only one
(e.g. '4s4s i 4s2s2s'): One that unpacks just the filtered fields
(e.g. '8x i 8x') and one that unpacks all the fields except the one
already created by the filter (e.g. '4s4s  4x  4s2s2s'). This solution
works very well and increases throughput by far. It however also
creates complexity in the code as we have to keep track and combine
field-values that came from the filtering-part with the ones unpacked
during inspection-part (we don't want to simply unpack twice).

I'd like to propose an enhancement to the struct module that should
solve this dilemma and ask for your comments.

The function s_unpack_internal() inside _struct.c currently unpacks
all values from the buffer-object passed to it and returns a tuple
holding these values. Instead, the function could create a tuple-like
object that holds a reference to it's own Struct-object (which holds
the format) and a copy of the memory it is supposed to unpack. This
object allows access to the unpacked values through the sequence
protocol, basically unpacking the fields if - and only if - accessed
through sq_item (e.g. foo = struct.unpack('2s2s', 'abcd'); foo[0] ==
'ab'). The object can also unpack all fields only once (as all
unpacked objects are immutable, we can hold references to them and
return these instead once known). This approach is possible because
there are no further error conditions inside the unpacking-functions
that we would *have* to deal with at the time .unpack() is called; in
other words: Unpacking can't fail if the format-string's syntax had
been correct and can therefor be deferred (while packing can't).

I understand that this may seem like a single-case-optimization. We
can however assume that most people will benefit from the new behavior
unknowingly while everyone else takes now harm: The object mimicking
the otherwise returned tuple is immutable (therefor it's not suddenly
part of GC) and the memory overhead caused by holding references to
the original memory a little longer (reclaimed after the result
becomes unreachable) should be comparable to the memory used by
unneeded fields (reclaimed directly after creation).

I'd like to hear your thoughts and am perfectly willing to provide a
patch if it has a chance of inclusion.


Best regards
Lukas

From regebro at gmail.com  Sun Jun 12 18:52:53 2011
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 12 Jun 2011 18:52:53 +0200
Subject: [Python-Dev] Implement Aspect-oriented programming
In-Reply-To: <BANLkTin46+QbM3ScxC=vw0z-GpU_ZqcvDw@mail.gmail.com>
References: <BANLkTin46+QbM3ScxC=vw0z-GpU_ZqcvDw@mail.gmail.com>
Message-ID: <BANLkTikTKJ8nO0Fp5osqrLPpu74amipA2Q@mail.gmail.com>

On Sat, Jun 11, 2011 at 10:29, Jiawei Li <jiawei.h.li at gmail.com> wrote:
> For example, the logging module is not very useful right now, as it requires
> sprinkling small one-liners all over my code - not exactly ideal.
> Why not take a page from aspect-oriented programming and allow for injection
> of code with point cuts?

I'm not sure why you would say this isn't allowed already...

-- 
Lennart Regebro: http://regebro.wordpress.com/
Porting to Python 3: http://python3porting.com/

From phd at phdru.name  Sun Jun 12 18:54:23 2011
From: phd at phdru.name (Oleg Broytman)
Date: Sun, 12 Jun 2011 20:54:23 +0400
Subject: [Python-Dev] Implement Aspect-oriented programming
In-Reply-To: <BANLkTin46+QbM3ScxC=vw0z-GpU_ZqcvDw@mail.gmail.com>
References: <BANLkTin46+QbM3ScxC=vw0z-GpU_ZqcvDw@mail.gmail.com>
Message-ID: <20110612165423.GA22939@iskra.aviel.ru>

Hi! This mailing list is to work on developing Python (discussing bugs
and patches). There is python-ideas mailing list to discuss possible
future improvements.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From ethan at stoneleaf.us  Sun Jun 12 19:18:24 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sun, 12 Jun 2011 10:18:24 -0700
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
References: <4DD2C2A5.3080403@stoneleaf.us>
	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>
	<4DD2D89D.4000303@stoneleaf.us>
	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>
	<4DD2F661.2050005@stoneleaf.us>
	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>
	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
Message-ID: <4DF4F4E0.5060003@stoneleaf.us>

Guido van Rossum wrote:
> On Thu, May 19, 2011 at 1:43 AM, Nick Coghlan wrote:
>> Proposals to address this include:
>> - introduce a "character" literal to allow c'a' as an alternative to ord('a')
> 
> -1; the result is not a *character* but an integer. I'm personally
> favoring using b'a'[0] and possibly hiding this in a constant
> definition.

Using this method, my code now looks like:

# constants

EOH  = b'\r'[0]
CHAR = b'C'[0]
DATE = b'D'[0]
FLOAT = b'F'[0]
INT = b'I'[0]
LOGICAL = b'L'[0]
MEMO = b'M'[0]
NUMBER = b'N'[0]

This is not beautiful code.

~Ethan~

From raymond.hettinger at gmail.com  Sun Jun 12 19:27:06 2011
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sun, 12 Jun 2011 10:27:06 -0700
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
Message-ID: <6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>


On Jun 12, 2011, at 8:29 AM, Lukas Lueg wrote:

> Hi.
> 
> We extensively use the struct module to crunch large amounts of binary
> data. There are basically two operations for us that only seem to the
> naked eye as one: Filtering (see if certain fields have certain
> values, throw everything away if not) and inspection (care about all
> the fields' values). The filtering-part is very important as most of
> the binary data can actually be thrown away and never have to be
> inspected any further. When thinking about how to increase
> performance, one thought was that a lot of objects are generated by
> the struct module that we never need: Unpacking six fields in order to
> look at one and then throwing everything away is inefficient
> concerning the five other fields. It also makes filtering and
> inspecting basically the same operation regarding the (slow) unpacking
> of values so we don't really benefit from filtering. This is a huge
> problem when crunching gigabytes of data and creating millions of
> fields.
> 
> One solution to this is using two format-strings instead of only one
> (e.g. '4s4s i 4s2s2s'): One that unpacks just the filtered fields
> (e.g. '8x i 8x') and one that unpacks all the fields except the one
> already created by the filter (e.g. '4s4s  4x  4s2s2s'). This solution
> works very well and increases throughput by far. I

This is what people normally do (unpack just the values they need,
when they need them).

> t however also
> creates complexity in the code as we have to keep track and combine
> field-values that came from the filtering-part with the ones unpacked
> during inspection-part (we don't want to simply unpack twice).
> 
> I'd like to propose an enhancement to the struct module that should
> solve this dilemma and ask for your comments.
> 
> The function s_unpack_internal() inside _struct.c currently unpacks
> all values from the buffer-object passed to it and returns a tuple
> holding these values. Instead, the function could create a tuple-like
> object that holds a reference to it's own Struct-object (which holds
> the format) and a copy of the memory it is supposed to unpack. This
> object allows access to the unpacked values through the sequence
> protocol, basically unpacking the fields if - and only if - accessed
> through sq_item (e.g. foo = struct.unpack('2s2s', 'abcd'); foo[0] ==
> 'ab'). The object can also unpack all fields only once (as all
> unpacked objects are immutable, we can hold references to them and
> return these instead once known). This approach is possible because
> there are no further error conditions inside the unpacking-functions
> that we would *have* to deal with at the time .unpack() is called; in
> other words: Unpacking can't fail if the format-string's syntax had
> been correct and can therefor be deferred (while packing can't).
> 
> I understand that this may seem like a single-case-optimization.

Yes, it does.

> We
> can however assume that most people will benefit from the new behavior
> unknowingly while everyone else takes now harm: The object mimicking
> the otherwise returned tuple is immutable (therefor it's not suddenly
> part of GC) and the memory overhead caused by holding references to
> the original memory a little longer (reclaimed after the result
> becomes unreachable) should be comparable to the memory used by
> unneeded fields (reclaimed directly after creation).
> 
> I'd like to hear your thoughts and am perfectly willing to provide a
> patch if it has a chance of inclusion.

The problem you're trying to solve isn't unique to structs.
That's why we get periodic requests for ropes-like behaviors
for strings for example.  Someone could also request lazy
extraction of fields in regular expressions or lazy parses of
json objects.  

I don't think there is a net win from adding complexity to the struct
module.  Introducing lazy behaviors creates its own overhead
that would compete with code optimized using the traditional
approach (unpack what you need, when you need it).  Also,
the new behaviors add to the cognitive load when learning
and remembering how to use this module.  

In general, Python has opted for the most straight-forward, least
magical implementations of object (why we use array based
lists instead of blist for example).  The are exceptions 
such as Python 3's version of super() but this isn't the norm.

I do suggest that you publish your code as a third-party module
to make the optional available and to validate whether there
is any real interest in this.


Raymond





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110612/77804cee/attachment.html>

From martin at v.loewis.de  Sun Jun 12 19:34:42 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 12 Jun 2011 19:34:42 +0200
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <4DF4F4E0.5060003@stoneleaf.us>
References: <4DD2C2A5.3080403@stoneleaf.us>	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>	<4DD2D89D.4000303@stoneleaf.us>	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>	<4DD2F661.2050005@stoneleaf.us>	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us>
Message-ID: <4DF4F8B2.5050802@v.loewis.de>

> # constants
> 
> EOH  = b'\r'[0]
> CHAR = b'C'[0]
> DATE = b'D'[0]
> FLOAT = b'F'[0]
> INT = b'I'[0]
> LOGICAL = b'L'[0]
> MEMO = b'M'[0]
> NUMBER = b'N'[0]
> 
> This is not beautiful code.

In this case, I think the intent would be better captured with

def ASCII(c):
    return c.encode('ascii')

EOH     = ASCII('\r') # 0D
CHAR    = ASCII('C')  # 43
DATE    = ASCII('D')  # 44
FLOAT   = ASCII('F')  # 46
INT     = ASCII('I')  # 49
LOGICAL = ASCII('L')  # 4C
MEMO    = ASCII('M')  # 4D
NUMBER  = ASCII('N')  # 4E

This expresses the intent that a) these are really byte values,
not characters, and b) the specific choice of byte values was motivated
by ASCII.

Regards,
Martin

From hagen at zhuliguan.net  Sun Jun 12 19:46:22 2011
From: hagen at zhuliguan.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=)
Date: Sun, 12 Jun 2011 19:46:22 +0200
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <4DF4F4E0.5060003@stoneleaf.us>
References: <4DD2C2A5.3080403@stoneleaf.us>	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>	<4DD2D89D.4000303@stoneleaf.us>	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>	<4DD2F661.2050005@stoneleaf.us>	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us>
Message-ID: <4DF4FB6E.2030009@zhuliguan.net>

> EOH  = b'\r'[0]
> CHAR = b'C'[0]
> DATE = b'D'[0]
> FLOAT = b'F'[0]
> INT = b'I'[0]
> LOGICAL = b'L'[0]
> MEMO = b'M'[0]
> NUMBER = b'N'[0]
> 
> This is not beautiful code.

You still have the alternative

EOH = ord('\r')
CHAR = ord('C')
...

which looks fine to me.

Cheers,
Hagen


From benjamin at python.org  Sun Jun 12 19:57:20 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 12 Jun 2011 12:57:20 -0500
Subject: [Python-Dev] [RELEASE] Python 2.7.2
Message-ID: <BANLkTi=jPF_bxWm2aNOnvN63Tgi8DELVhA@mail.gmail.com>

On behalf of the Python development team, I'm rosy to announce the immediate
availability of Python 2.7.2.

Since the release candidate 2 weeks ago, there have been 2 changes:
1. pyexpat.__version__ has be changed to be the Python version. 2. A regression
from 3.1.3 in the handling of comments in the netrc module has been
resolved. (see issue #12009).

2.7.2 is the second in bugfix release for the Python 2.7 series. 2.7 is the last
major verison of the 2.x line and will be receiving only bug fixes while new
feature development focuses on 3.x.

The 2.7 series includes many features that were first released in Python
3.1. The faster io module, the new nested with statement syntax, improved float
repr, set literals, dictionary views, and the memoryview object have been
backported from 3.1. Other features include an ordered dictionary
implementation, unittests improvements, a new sysconfig module, auto-numbering
of fields in the str/unicode format method, and support for ttk Tile in Tkinter.
For a more extensive list of changes in 2.7, see
http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python
distribution.

To download Python 2.7.2 visit:

     http://www.python.org/download/releases/2.7.1/

The 2.7.2 changelog is at:

     http://hg.python.org/cpython/raw-file/eb3c9b74884c/Misc/NEWS

2.7 documentation can be found at:

     http://docs.python.org/2.7/

This is a production release, please report any bugs to

     http://bugs.python.org/


Enjoy and for those in the northern hemisphere, have a nice summer!

--
Benjamin Peterson
Release Manager
benjamin at python.org
(on behalf of the entire python-dev team and 2.7.2's contributors)

From benjamin at python.org  Sun Jun 12 19:58:47 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 12 Jun 2011 12:58:47 -0500
Subject: [Python-Dev] [RELEASED] Python 3.1.4
Message-ID: <BANLkTin84-BP2kNGvK0i4NVN3O9OEEgg8Q@mail.gmail.com>

On behalf of the Python development team, I'm sanguine to announce a release
candidate for the fourth bugfix release for the Python 3.1 series, Python 3.1.4.

Since the 3.1.4 release candidate 2 weeks ago, there have been three changes:
1. test_zipfile has been fixed on systems with an ASCII filesystem
encoding. 2. pyexpat.__version__ has be changed to be the Python version. 3. A
regression from 2.7.1 in the handling of comments in the netrc module has been
resolved. (see issue #12009).

3.1.4 will the last bug fix release in the 3.1 series before 3.1. After 3.1.4,
3.1 will be in security-only fix mode.

The Python 3.1 version series focuses on the stabilization and optimization of
the features and changes that Python 3.0 introduced.  For example, the new I/O
system has been rewritten in C for speed.  File system APIs that use unicode
strings now handle paths with undecodable bytes in them. Other features include
an ordered dictionary implementation, a condensed syntax for nested with
statements, and support for ttk Tile in Tkinter.  For a more extensive list of
changes in 3.1, see http://doc.python.org/3.1/whatsnew/3.1.html or Misc/NEWS in
the Python distribution.

This is a production release. To download Python 3.1.4 visit:

     http://www.python.org/download/releases/3.1.4/

A list of changes in 3.1.4 can be found here:

     http://hg.python.org/cpython/raw-file/feae9f9e9f30/Misc/NEWS

The 3.1 documentation can be found at:

     http://docs.python.org/3.1

Bugs can always be reported to:

     http://bugs.python.org


Enjoy and be merry!

--
Benjamin Peterson
Release Manager
benjamin at python.org
(on behalf of the entire python-dev team and 3.1.4's contributors)

From lukas.lueg at googlemail.com  Sun Jun 12 19:43:39 2011
From: lukas.lueg at googlemail.com (Lukas Lueg)
Date: Sun, 12 Jun 2011 19:43:39 +0200
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
	<6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
Message-ID: <BANLkTi=XDPazB1cFJp7=Fx5+9QgB+H60ng@mail.gmail.com>

> This is what people normally do (unpack just the values they need,
> when they need them).
Due to the fact that there hundreds of format-strings which
dynamically compiled from a more verbose language at runtime, we will
have significant complexity in the code in order to generate format
strings that parse just the fields that are needed for filtering. It's
not just put-a-string-here-and-there.

> I don't think there is a net win from adding complexity to the struct
> module. ?Introducing lazy behaviors creates its own overhead
> that would compete with code optimized using the traditional
> approach (unpack what you need, when you need it). ?Also,
> the new behaviors add to the cognitive load when learning
> and remembering how to use this module.

The complexity is very well handled. Remember that the interface to
the module does not change at all and the documentation would be
exactly the same. There is no special case introduced here the user
has to know about. I also think this case has very little black magic
in it since we are dealing only with immutable objects and do not have
delayed error conditions (both usually being the primary source of
headaches when introducing lazy behavior).

From tjreedy at udel.edu  Sun Jun 12 20:28:19 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 12 Jun 2011 14:28:19 -0400
Subject: [Python-Dev] [RELEASE] Python 2.7.2
In-Reply-To: <BANLkTi=jPF_bxWm2aNOnvN63Tgi8DELVhA@mail.gmail.com>
References: <BANLkTi=jPF_bxWm2aNOnvN63Tgi8DELVhA@mail.gmail.com>
Message-ID: <it30g1$7k9$1@dough.gmane.org>

On 6/12/2011 1:57 PM, Benjamin Peterson wrote:

>
> To download Python 2.7.2 visit:
>
>       http://www.python.org/download/releases/2.7.1/
That should be
http://www.python.org/download/releases/2.7.2/
-- 
Terry Jan Reedy


From p.f.moore at gmail.com  Sun Jun 12 21:31:59 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 12 Jun 2011 20:31:59 +0100
Subject: [Python-Dev] [RELEASED] Python 3.1.4
In-Reply-To: <BANLkTin84-BP2kNGvK0i4NVN3O9OEEgg8Q@mail.gmail.com>
References: <BANLkTin84-BP2kNGvK0i4NVN3O9OEEgg8Q@mail.gmail.com>
Message-ID: <BANLkTin2JBbk_5CBnYCPd3H4cVON-KmX8w@mail.gmail.com>

On 12 June 2011 18:58, Benjamin Peterson <benjamin at python.org> wrote:
> On behalf of the Python development team, I'm sanguine to announce a release
> candidate for the fourth bugfix release for the Python 3.1 series, Python 3.1.4.

Is this actually a RC, or is that a typo?
Paul.

From benjamin at python.org  Sun Jun 12 21:52:42 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 12 Jun 2011 14:52:42 -0500
Subject: [Python-Dev] [RELEASED] Python 3.1.4
In-Reply-To: <BANLkTin2JBbk_5CBnYCPd3H4cVON-KmX8w@mail.gmail.com>
References: <BANLkTin84-BP2kNGvK0i4NVN3O9OEEgg8Q@mail.gmail.com>
	<BANLkTin2JBbk_5CBnYCPd3H4cVON-KmX8w@mail.gmail.com>
Message-ID: <BANLkTikhy4_5nLR23AatbwoO44+54buPJA@mail.gmail.com>

2011/6/12 Paul Moore <p.f.moore at gmail.com>:
> On 12 June 2011 18:58, Benjamin Peterson <benjamin at python.org> wrote:
>> On behalf of the Python development team, I'm sanguine to announce a release
>> candidate for the fourth bugfix release for the Python 3.1 series, Python 3.1.4.
>
> Is this actually a RC, or is that a typo?

That is a typo. This is a final release!

> Paul.
>



-- 
Regards,
Benjamin

From tjreedy at udel.edu  Sun Jun 12 22:31:47 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 12 Jun 2011 16:31:47 -0400
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
Message-ID: <it37nk$dvu$1@dough.gmane.org>

On 6/12/2011 11:29 AM, Lukas Lueg wrote:

This sort of speculative idea might fit the python-ideas list better.

[Summary: we often need to extract a field or two from a binary record 
in order to decide whether to toss it or unpack it all and process.]

> One solution to this is using two format-strings instead of only one
> (e.g. '4s4s i 4s2s2s'): One that unpacks just the filtered fields
> (e.g. '8x i 8x') and one that unpacks all the fields except the one
> already created by the filter (e.g. '4s4s  4x  4s2s2s'). This solution
> works very well and increases throughput by far. It however also
> creates complexity in the code as we have to keep track and combine
> field-values that came from the filtering-part with the ones unpacked
> during inspection-part (we don't want to simply unpack twice).

With just 1 or 2 filter fields, and very many other fields, I would just 
unpack everything, including the filter field. I expect the extra time 
to do that would be comparalbe to the extra time to combine. It 
certainly would make your code easier. I suspect you could write a 
function to create the filter field only format by field number from the 
everything format.

> I'd like to propose an enhancement to the struct module that should
> solve this dilemma and ask for your comments.
>
> The function s_unpack_internal() inside _struct.c currently unpacks
> all values from the buffer-object passed to it and returns a tuple
> holding these values. Instead, the function could create a tuple-like
> object that holds a reference to it's own Struct-object (which holds
> the format) and a copy of the memory it is supposed to unpack. This
> object allows access to the unpacked values through the sequence
> protocol, basically unpacking the fields if - and only if - accessed
> through sq_item (e.g. foo = struct.unpack('2s2s', 'abcd'); foo[0] ==
> 'ab'). The object can also unpack all fields only once (as all
> unpacked objects are immutable, we can hold references to them and
> return these instead once known). This approach is possible because
> there are no further error conditions inside the unpacking-functions
> that we would *have* to deal with at the time .unpack() is called; in
> other words: Unpacking can't fail if the format-string's syntax had
> been correct and can therefor be deferred (while packing can't).
>
> I understand that this may seem like a single-case-optimization.

Yep.

> We
> can however assume that most people will benefit from the new behavior
> unknowingly while everyone else takes now harm:

I will not assume that without code and timings. I would expect that 
unpacking one field at a time would take longer than all at once. To me, 
this is the sort of thing that should be written, listed on PyPI, and 
tested by multiple users on multiple systems first.

-- 
Terry Jan Reedy


From benjamin at python.org  Sun Jun 12 22:37:15 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 12 Jun 2011 15:37:15 -0500
Subject: [Python-Dev] is anyone using Misc/RPM?
Message-ID: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>

If no one is using it, I'd like to delete it. I also don't think we
should be in business of distributing distribution specific files.

-- 
Regards,
Benjamin

From martin at v.loewis.de  Sun Jun 12 23:39:39 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 12 Jun 2011 23:39:39 +0200
Subject: [Python-Dev] is anyone using Misc/RPM?
In-Reply-To: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>
References: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>
Message-ID: <4DF5321B.4000603@v.loewis.de>

Am 12.06.2011 22:37, schrieb Benjamin Peterson:
> I also don't think we
> should be in business of distributing distribution specific files.

I disagree. We certainly include PCbuild/*.vcproj, and Tools/msi,
which are also "distribution-specific". Likewise, we have plenty
of OSX-specific files (including special-casing for specific releases
thereof).

So having an RPM spec file in the source doesn't sound bad to me.
Of course, if it's unmaintained (in the sense that it doesn't
actually work), I could agree to have it deleted. Make sure
Sean Reifschneider doesn't object.

People are apparently using it - at least, they report bugs
against it:

http://bugs.python.org/issue5776
http://bugs.python.org/issue5063

Regards,
Martin

From benjamin at python.org  Sun Jun 12 23:44:55 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 12 Jun 2011 16:44:55 -0500
Subject: [Python-Dev] is anyone using Misc/RPM?
In-Reply-To: <4DF5321B.4000603@v.loewis.de>
References: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>
	<4DF5321B.4000603@v.loewis.de>
Message-ID: <BANLkTi=XYh9FT4jqZK5D2Q-b5cLQ2JPsuQ@mail.gmail.com>

2011/6/12 "Martin v. L?wis" <martin at v.loewis.de>:
> Am 12.06.2011 22:37, schrieb Benjamin Peterson:
>> I also don't think we
>> should be in business of distributing distribution specific files.
>
> I disagree. We certainly include PCbuild/*.vcproj, and Tools/msi,
> which are also "distribution-specific". Likewise, we have plenty
> of OSX-specific files (including special-casing for specific releases
> thereof).

I should qualify: We shouldn't distribute distribution-specific files
for which we don't provide binaries.



-- 
Regards,
Benjamin

From greg.ewing at canterbury.ac.nz  Mon Jun 13 00:12:11 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 13 Jun 2011 10:12:11 +1200
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <4DF4F4E0.5060003@stoneleaf.us>
References: <4DD2C2A5.3080403@stoneleaf.us>
	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>
	<4DD2D89D.4000303@stoneleaf.us>
	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>
	<4DD2F661.2050005@stoneleaf.us>
	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>
	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us>
Message-ID: <4DF539BB.4070409@canterbury.ac.nz>

Guido van Rossum wrote:

>> On Thu, May 19, 2011 at 1:43 AM, Nick Coghlan wrote:
>>
>>> Proposals to address this include:
>>> - introduce a "character" literal to allow c'a' as an alternative to 
>>> ord('a')
>
> -1; the result is not a *character* but an integer.

Would you be happier if it were spelled i'a' instead?

-- 
Greg

From greg.ewing at canterbury.ac.nz  Mon Jun 13 00:18:53 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 13 Jun 2011 10:18:53 +1200
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
	<6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
Message-ID: <4DF53B4D.1000607@canterbury.ac.nz>

Raymond Hettinger wrote:

> The problem you're trying to solve isn't unique to structs.
> That's why we get periodic requests for ropes-like behaviors

I don't think he's asking for rope-like behaviour here.
Rather, he's asking for iterator-like or view-like
behaviour -- for the same reasons we have both zip() and
izip(), for example, or that dict.items() became a
view in Py3. It seems like a reasonable request to me.

However, I'm wondering whether the ctypes module might
already provide what he wants.

-- 
Greg

From raymond.hettinger at gmail.com  Mon Jun 13 00:56:10 2011
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Sun, 12 Jun 2011 15:56:10 -0700
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <4DF53B4D.1000607@canterbury.ac.nz>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
	<6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
	<4DF53B4D.1000607@canterbury.ac.nz>
Message-ID: <E784DF2D-A8F9-4D81-998E-77F2905C9AB2@gmail.com>


On Jun 12, 2011, at 3:18 PM, Greg Ewing wrote:

> Raymond Hettinger wrote:
> 
>> The problem you're trying to solve isn't unique to structs.
>> That's why we get periodic requests for ropes-like behaviors
> 
> I don't think he's asking for rope-like behaviour here.

How would you describe the creation of a lazy result
that keeps a reference to the underlying buffer?
That is my understanding of how ropes work.


Raymond


From stephen at xemacs.org  Mon Jun 13 03:42:43 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 13 Jun 2011 10:42:43 +0900
Subject: [Python-Dev] is anyone using Misc/RPM?
In-Reply-To: <BANLkTi=XYh9FT4jqZK5D2Q-b5cLQ2JPsuQ@mail.gmail.com>
References: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>
	<4DF5321B.4000603@v.loewis.de>
	<BANLkTi=XYh9FT4jqZK5D2Q-b5cLQ2JPsuQ@mail.gmail.com>
Message-ID: <877h8qmrqk.fsf@uwakimon.sk.tsukuba.ac.jp>

Benjamin Peterson writes:

 > I should qualify: We shouldn't distribute distribution-specific files
 > for which we don't provide binaries.

Probably it belongs in a "contrib" area of the tree, but one of the
things I find really annoying about distros is the way they refuse to
use my perfectly good locally built Python (XEmacs, Mailman, Django,
Zope, ...).  Having the magic incantation to persuade them that the
locally built software satisfies dependencies in the source itself is
very convenient.

In fact, even if you *do* provide binaries it may be useful to have
both the "provided" installer configuration (which may require things
like DBMSes, perhaps several of them) and a bare-bones config for
DIYers to use.  (Violates TOOWTDI, I know, but PBP sometimes.)


From stephen at xemacs.org  Mon Jun 13 03:52:14 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 13 Jun 2011 10:52:14 +0900
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <4DF4F4E0.5060003@stoneleaf.us>
References: <4DD2C2A5.3080403@stoneleaf.us>
	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>
	<4DD2D89D.4000303@stoneleaf.us>
	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>
	<4DD2F661.2050005@stoneleaf.us>
	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>
	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us>
Message-ID: <8762oamrap.fsf@uwakimon.sk.tsukuba.ac.jp>

Ethan Furman writes:

 > Using this method, my code now looks like:
 > 
 > # constants

[...]

 > This is not beautiful code.

Put mascara on a pig, and you have a pig with mascara on, not Bette
Davis.  I don't necessarily think you're doing anybody a service by
making the hack of using ASCII bytes as mnemonics more beautiful.  I
think Martin's version is as beautiful as this code should get.



From ncoghlan at gmail.com  Mon Jun 13 06:42:33 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2011 14:42:33 +1000
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <4DF4F4E0.5060003@stoneleaf.us>
References: <4DD2C2A5.3080403@stoneleaf.us>
	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>
	<4DD2D89D.4000303@stoneleaf.us>
	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>
	<4DD2F661.2050005@stoneleaf.us>
	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>
	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us>
Message-ID: <BANLkTinOc51SPCJCMzmonDRN=CtT2xKRLQ@mail.gmail.com>

On Mon, Jun 13, 2011 at 3:18 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
>
> This is not beautiful code.

Agreed, but:

EOH, CHAR, DATE, FLOAT, INT, LOGICAL, MEMO, NUMBER = b'\rCDFILMN'

is a shorter way to write the same thing.

Going two per line makes it easier to mentally map the characters:

EOH, CHAR = b'\rC'
DATE, FLOAT = b'DF'
INT, LOGICAL = b'IL'
MEMO, NUMBER = b'MN'

Or, as a variant on Martin's solution:

FORMAT_CHARS = dict(
  EOH = '\r',
  CHAR= 'C',
  DATE = 'D',
  FLOAT = 'F',
  INT = 'I',
  LOGICAL = 'L',
  MEMO = 'M',
  NUMBER = 'N'
)

FORMAT_CODES = {name : char.encode('ascii') for name, char in FORMAT_CHARS}
globals().update(FORMAT_CODES)

Sure, there's no "one obvious way" at this stage, but that's because
we don't know yet if there even *should* be an obvious way to do this
(as conflating text and binary data is a bad idea in principle). By
not blessing any one way of handling the situation, we give
alternative solutions time to evolve naturally. If one turns out to be
clearly superior to the decode/process/encode cycle then hopefully
that will become clear at some point in the future.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Jun 13 07:34:33 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2011 15:34:33 +1000
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <it37nk$dvu$1@dough.gmane.org>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
	<it37nk$dvu$1@dough.gmane.org>
Message-ID: <BANLkTin-Ym0C4Ah3OKgs2K0Pxak99Ztodg@mail.gmail.com>

On Mon, Jun 13, 2011 at 6:31 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> With just 1 or 2 filter fields, and very many other fields, I would just
> unpack everything, including the filter field. I expect the extra time to do
> that would be comparalbe to the extra time to combine. It certainly would
> make your code easier. I suspect you could write a function to create the
> filter field only format by field number from the everything format.

Indeed, while the "filter format" part makes sense to me, the decision
to go with field combination rather than just extracting the filter
fields a second time isn't such an obvious improvement.

OTOH, it also seems like this is something that could be published as
a cookbook recipe that generated the appropriate filtered formats on
the fly from an existing struct definition. So given format "a b c d
e", it could either extract each field individually, or else be asked
to generate specific formats and their complements (e.g, asking for
the 2nd and 3rd field could return a 2-tuple of formats "x b c x x"
and "a x x c d e").

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ocean-city at m2.ccsnet.ne.jp  Mon Jun 13 07:36:48 2011
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Mon, 13 Jun 2011 14:36:48 +0900
Subject: [Python-Dev] Fwd: [Python-committers] Pulling from contributors
	repositories
Message-ID: <4DF5A1F0.2030207@m2.ccsnet.ne.jp>

I've read the Python-committers thread "Pulling from contributors 
repositories", which is about version control system. It seems there are 
two main issues, linear (cleaner) history on pushing, and NEWS merging. 
I'm newby of bazaar, but it seems to have a solution for first issue.

$ bzr checkout /repo/trunk

$ bzr merge /repo/feature-a

$ bzr revert --forget-merges

$ bzr push

See 
http://doc.bazaar.canonical.com/latest/en/user-guide/adv_merging.html#merging-without-parents


From victor.stinner at haypocalc.com  Mon Jun 13 10:44:22 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 13 Jun 2011 10:44:22 +0200
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
	<6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
Message-ID: <1307954662.22317.6.camel@marge>

Le dimanche 12 juin 2011 ? 10:27 -0700, Raymond Hettinger a ?crit :
> I do suggest that you publish your code as a third-party module
> to make the optional available and to validate whether there
> is any real interest in this.

See the Hachoir project: it is a lazy parser supporting sub-structures
(whereas struct only supports flat structures). It is difficult to
implement a lazy parser: I chose to use Python generators to implement
them. Hachoir should not enter Python standard library: it evoles too
fast and it is too big (60K lines of Python).

See also:

bdec: http://www.protocollogic.com/
Construct: http://construct.wikispaces.com/
FileAlyzer: http://www.safer-networking.org/fr/filealyzer/index.html
DataWorkshop: http://www.dataworkshop.de/

DataWorkshop project is dead since 2005. I don't remember if FileAlyzer
is a free software or not.

I agree with Raymond: struct module should be kept simple, and if you
want a lazy parser: it should be a third party project.

Victor


From vinay_sajip at yahoo.co.uk  Mon Jun 13 13:47:33 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Jun 2011 11:47:33 +0000 (UTC)
Subject: [Python-Dev] In-Python virtualisation and packaging
Message-ID: <loom.20110613T132530-565@post.gmane.org>

Back in March this year, Carl Meyer did some work to see if it was feasible
to bring virtualenv functionality into Python [1] (code at [2]).

Carl's changes were to Python code only, which was almost but not quite enough.
I built on his changes with updates to C code in getpath.c/getpathp.c, and my
code is at [3]. I've kept it synchronised with the core cpython repo, including
the recently committed packaging modules.

While there are issues to work through such as dealing with source builds (and
no doubt plenty of others), it now seems possible to create virtual
environments and install stuff into them using just the stdlib (modulo
currently needing Distribute for the packages which don't yet support
setup.cfg-based packaging, but it's all done automatically for a user). So you
can do e.g.

$ python3.3 -m virtualize /tmp/venv
$ source /tmp/venv/bin/activate.sh
$ pysetup3 install Mako

and so on. A log of early experiments, which seems reasonably promising, is at
[4].

Do people agree that it may be fitting, proper and timely to bring
virtualisation into Python, and are there any fundamental flaws anyone can see
with the approach used?

If people want to experiment with this code without cloning and building, I
created a Debian package using checkinstall, which can be installed using

sudo dpkg -i pythonv_3.3-1_i386.deb

and removed using

sudo dpkg -r pythonv

I can make this Debian package available for download, if anyone wants it.

Regards,

Vinay Sajip

[1] http://mail.python.org/pipermail/distutils-sig/2011-March/017519.html
[2] https://bitbucket.org/carljm/cpythonv
[3] https://bitbucket.org/vinay.sajip/pythonv
[4] https://gist.github.com/1022601


From fuzzyman at voidspace.org.uk  Mon Jun 13 13:55:52 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 13 Jun 2011 12:55:52 +0100
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <loom.20110613T132530-565@post.gmane.org>
References: <loom.20110613T132530-565@post.gmane.org>
Message-ID: <4DF5FAC8.4090606@voidspace.org.uk>

On 13/06/2011 12:47, Vinay Sajip wrote:
> Back in March this year, Carl Meyer did some work to see if it was feasible
> to bring virtualenv functionality into Python [1] (code at [2]).
>
> Carl's changes were to Python code only, which was almost but not quite enough.
> I built on his changes with updates to C code in getpath.c/getpathp.c, and my
> code is at [3]. I've kept it synchronised with the core cpython repo, including
> the recently committed packaging modules.
>
> While there are issues to work through such as dealing with source builds (and
> no doubt plenty of others), it now seems possible to create virtual
> environments and install stuff into them using just the stdlib (modulo
> currently needing Distribute for the packages which don't yet support
> setup.cfg-based packaging, but it's all done automatically for a user). So you
> can do e.g.
>
> $ python3.3 -m virtualize /tmp/venv
> $ source /tmp/venv/bin/activate.sh
> $ pysetup3 install Mako
>
> and so on. A log of early experiments, which seems reasonably promising, is at
> [4].
>
> Do people agree that it may be fitting, proper and timely to bring
> virtualisation into Python, and are there any fundamental flaws anyone can see
> with the approach used?

It would certainly need a PEP.

There are two options:

Bring the full functionality into the standard library so that Python 
supports virtual environments out of the box. As is the case with adding 
anything to the standard library it will then be impossible to add 
features to the virtualization support in Python 3.3 once 3.3 is 
released - new features will go into 3.4.

Add only the minimal changes required to support a third-party virtual 
environment tool.

Virtual environments are phenomenally useful, so I would support having 
the full tool in the standard library, but it does raise maintenance and 
development issues.

Don't forget windows support! ;-)

All the best,

Michael Foord


> If people want to experiment with this code without cloning and building, I
> created a Debian package using checkinstall, which can be installed using
>
> sudo dpkg -i pythonv_3.3-1_i386.deb
>
> and removed using
>
> sudo dpkg -r pythonv
>
> I can make this Debian package available for download, if anyone wants it.
>
> Regards,
>
> Vinay Sajip
>
> [1] http://mail.python.org/pipermail/distutils-sig/2011-March/017519.html
> [2] https://bitbucket.org/carljm/cpythonv
> [3] https://bitbucket.org/vinay.sajip/pythonv
> [4] https://gist.github.com/1022601
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From ncoghlan at gmail.com  Mon Jun 13 14:09:14 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2011 22:09:14 +1000
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <4DF5FAC8.4090606@voidspace.org.uk>
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
Message-ID: <BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>

On Mon, Jun 13, 2011 at 9:55 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Virtual environments are phenomenally useful, so I would support having the
> full tool in the standard library, but it does raise maintenance and
> development issues.
>
> Don't forget windows support! ;-)

Given that it is desirable for tools like virtualenv to support *old*
versions of Python on *new* versions of operating systems, it seems to
me that there is an inherent element of their feature set that makes
including the whole tool questionable.

OTOH, it may make sense to have a baseline tool provided innately, but
provide the appropriate third party hooks to allow alternative tools
to evolve independently of the stdlib.

How well does the regression test suite cope when run inside such a
virtualised environment?

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Mon Jun 13 14:22:32 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Jun 2011 12:22:32 +0000 (UTC)
Subject: [Python-Dev] In-Python virtualisation and packaging
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
Message-ID: <loom.20110613T141642-955@post.gmane.org>

Michael Foord <fuzzyman <at> voidspace.org.uk> writes:

> It would certainly need a PEP.

Of course.

> There are two options:
> 
> Bring the full functionality into the standard library so that Python 
> supports virtual environments out of the box. As is the case with adding 
> anything to the standard library it will then be impossible to add 
> features to the virtualization support in Python 3.3 once 3.3 is 
> released - new features will go into 3.4.
> 
> Add only the minimal changes required to support a third-party virtual 
> environment tool.
>

Agreed. As I see it, the "minimal changes required" are everything in my fork
except for "virtualize.py", which was actually written as an external module
"pmv.py" - Poor Man's Virtualenv ;-)

Having it as a minimal implementation in the stdlib accords with "batteries
included", but even as it stands, virtualize.py does try to cater for
customisation. Firstly, there's a virtualizer_factory callable which can be
overridden for customisation. That's called to produce a virtualizer, whose
virtualize method is called with the target directory. The virtualize()
function in virtualize.py just does this set of steps. I can't claim to have
thought of everything, but it's a simple API which could have any number of
implementations, not just the default one in the Virtualizer class in
virtualize.py.

> Don't forget windows support! 

I haven't. Though I haven't tested the most recent changes on Windows yet, I
have tested the basic approach under Windows (the code doesn't rely on
symlinks, but rather, copies of executables/DLLs). (All Windows testing so far
has admittedly been using source builds rather than via a binary installer.)

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Mon Jun 13 14:50:01 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Jun 2011 12:50:01 +0000 (UTC)
Subject: [Python-Dev] In-Python virtualisation and packaging
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
	<BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>
Message-ID: <loom.20110613T142559-463@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> Given that it is desirable for tools like virtualenv to support *old*
> versions of Python on *new* versions of operating systems, it seems to
> me that there is an inherent element of their feature set that makes
> including the whole tool questionable.

You're right in terms of the current Python ecosystem and 3.x adoption, because
of course this approach requires support from Python itself in terms of its
site.py code. However, virtual environments have a utility beyond supporting
older Pythons on newer OSes, since another common use case is having different
library environments sandboxed from each other on different projects, even if
all those projects are using Python 3.3+. The virtualenv module does an
intricate bootstrapping dance which needs to accommodate each successive Python
version, so there's maintenance overhead that way, too. Carl Meyer, being a pip
and virtualenv maintainer, will probably have useful views on this.

> OTOH, it may make sense to have a baseline tool provided innately, but
> provide the appropriate third party hooks to allow alternative tools
> to evolve independently of the stdlib.

Yes - I'm thinking that what I've proposed is the baseline tool, and the
question is about what the virtualisation API needs to look like to allow
third-party tools to progress independently of the stdlib but in an
interoperable way (a bit like packaging, I suppose).

> How well does the regression test suite cope when run inside such a
> virtualised environment?

https://gist.github.com/1022705

325 tests OK.
5 tests failed:
    test_email test_importlib test_lib2to3 test_packaging
    test_sysconfig

test_importlib might be broken because I accidentally committed some changes to
marshal.c while working on #12291.

test_packaging fails because of #12313.
test_email fails for a similar reason - Makefile.pre.in is missing test_email in
LIBSUBDIRS.

test_sysconfig is probably failing because of changes I made, and I'm not sure
of test_lib2to3. I will investigate!

Regards,

Vinay Sajip


From solipsis at pitrou.net  Mon Jun 13 14:57:36 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 13 Jun 2011 14:57:36 +0200
Subject: [Python-Dev] In-Python virtualisation and packaging
References: <loom.20110613T132530-565@post.gmane.org>
Message-ID: <20110613145736.66bd8f79@pitrou.net>

On Mon, 13 Jun 2011 11:47:33 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> 
> $ python3.3 -m virtualize /tmp/venv
> $ source /tmp/venv/bin/activate.sh
> $ pysetup3 install Mako
> 
> and so on. A log of early experiments, which seems reasonably promising, is at
> [4].
> 
> Do people agree that it may be fitting, proper and timely to bring
> virtualisation into Python, and are there any fundamental flaws anyone can see
> with the approach used?

This sounds really great, and definitely needs a PEP so that we can ask
many questions :)

As a side-note, I think calling it "virtualization" is a recipe for
confusion.

Regards

Antoine.



From ncoghlan at gmail.com  Mon Jun 13 15:04:52 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2011 23:04:52 +1000
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <20110613145736.66bd8f79@pitrou.net>
References: <loom.20110613T132530-565@post.gmane.org>
	<20110613145736.66bd8f79@pitrou.net>
Message-ID: <BANLkTikH7YMPJER-S2cyGxyQw113xRMLLA@mail.gmail.com>

On Mon, Jun 13, 2011 at 10:57 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> As a side-note, I think calling it "virtualization" is a recipe for
> confusion.

Indeed, OS level virtualisation pretty much has a lock on that term.
"virtual environments" skates close to it but manages to avoid it well
enough to avoid confusion.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Jun 13 15:07:11 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2011 23:07:11 +1000
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <loom.20110613T142559-463@post.gmane.org>
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
	<BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>
	<loom.20110613T142559-463@post.gmane.org>
Message-ID: <BANLkTikqG-VWkaSTGH3bb5JACcVZVcOR_A@mail.gmail.com>

On Mon, Jun 13, 2011 at 10:50 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> You're right in terms of the current Python ecosystem and 3.x adoption, because
> of course this approach requires support from Python itself in terms of its
> site.py code. However, virtual environments have a utility beyond supporting
> older Pythons on newer OSes, since another common use case is having different
> library environments sandboxed from each other on different projects, even if
> all those projects are using Python 3.3+.

Yeah, even if the innate one struggles on later OS releases that
changed things in a backwards incompatible way, it will still be
valuable on the OS versions that are around at the time that version
of Python gets released.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Jun 13 15:09:00 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Jun 2011 23:09:00 +1000
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <loom.20110613T141642-955@post.gmane.org>
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
	<loom.20110613T141642-955@post.gmane.org>
Message-ID: <BANLkTi=55zvaGHY2o9knPQHFDBNfHc=GDQ@mail.gmail.com>

On Mon, Jun 13, 2011 at 10:22 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
>> Don't forget windows support!
>
> I haven't. Though I haven't tested the most recent changes on Windows yet, I
> have tested the basic approach under Windows (the code doesn't rely on
> symlinks, but rather, copies of executables/DLLs). (All Windows testing so far
> has admittedly been using source builds rather than via a binary installer.)

You should be able to use symlinks even on Windows these days
(although granted they won't on portable media that uses a non-symlink
friendly filesystem, regardless of OS).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Mon Jun 13 15:42:06 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Jun 2011 13:42:06 +0000 (UTC)
Subject: [Python-Dev] In-Python virtualisation and packaging
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
	<loom.20110613T141642-955@post.gmane.org>
	<BANLkTi=55zvaGHY2o9knPQHFDBNfHc=GDQ@mail.gmail.com>
Message-ID: <loom.20110613T153652-252@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> You should be able to use symlinks even on Windows these days
> (although granted they won't on portable media that uses a non-symlink
> friendly filesystem, regardless of OS).

Plus I'm not sure Windows XP supports true symlinks - I think you have to make
do with "junctions" a.k.a. "reparse points" which are more shambolic than
symbolic ;-) I know symlinks are available on Vista, Windows Server 2008 and
later, but XP is still very common.

Regards,

Vinay Sajip


From brian.curtin at gmail.com  Mon Jun 13 15:47:57 2011
From: brian.curtin at gmail.com (Brian Curtin)
Date: Mon, 13 Jun 2011 08:47:57 -0500
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <loom.20110613T153652-252@post.gmane.org>
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
	<loom.20110613T141642-955@post.gmane.org>
	<BANLkTi=55zvaGHY2o9knPQHFDBNfHc=GDQ@mail.gmail.com>
	<loom.20110613T153652-252@post.gmane.org>
Message-ID: <BANLkTikPwznv9x+5KkiJvzMX3r7ySL8_2g@mail.gmail.com>

On Mon, Jun 13, 2011 at 08:42, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:

> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
> > You should be able to use symlinks even on Windows these days
> > (although granted they won't on portable media that uses a non-symlink
> > friendly filesystem, regardless of OS).
>
> Plus I'm not sure Windows XP supports true symlinks - I think you have to
> make
> do with "junctions" a.k.a. "reparse points" which are more shambolic than
> symbolic ;-) I know symlinks are available on Vista, Windows Server 2008
> and
> later, but XP is still very common.


I don't think we have any stdlib support for junctions, although we could
certainly add it. In 3.2 we added symlink support for files and directories,
which as you say is a Vista and beyond feature.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110613/43bb168b/attachment.html>

From vinay_sajip at yahoo.co.uk  Mon Jun 13 15:47:06 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Jun 2011 13:47:06 +0000 (UTC)
Subject: [Python-Dev] In-Python virtualisation and packaging
References: <loom.20110613T132530-565@post.gmane.org>
	<20110613145736.66bd8f79@pitrou.net>
	<BANLkTikH7YMPJER-S2cyGxyQw113xRMLLA@mail.gmail.com>
Message-ID: <loom.20110613T154326-403@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> On Mon, Jun 13, 2011 at 10:57 PM, Antoine Pitrou <solipsis <at> pitrou.net>
> wrote:
> > As a side-note, I think calling it "virtualization" is a recipe for
> > confusion.
> 
> Indeed, OS level virtualisation pretty much has a lock on that term.
> "virtual environments" skates close to it but manages to avoid it well
> enough to avoid confusion.

Or as they involving encapsulating paths and libaries, perhaps we could call
them "capsules" ;-) though I think the term virtualenv is pretty entrenched now
in the Python community.

Regards,

Vinay Sajip



From barry at python.org  Mon Jun 13 17:02:24 2011
From: barry at python.org (Barry Warsaw)
Date: Mon, 13 Jun 2011 11:02:24 -0400
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <loom.20110613T132530-565@post.gmane.org>
References: <loom.20110613T132530-565@post.gmane.org>
Message-ID: <20110613110224.05d6962e@neurotica.wooz.org>

On Jun 13, 2011, at 11:47 AM, Vinay Sajip wrote:

>Do people agree that it may be fitting, proper and timely to bring
>virtualisation into Python, and are there any fundamental flaws anyone can see
>with the approach used?

Yes, absolutely.  We'll hash out the details when the PEP is published, and
bikeshed on all the terminology, but I really think this would be a very
powerful addition to the standard library, so +1.  Hopefully, the maintenance
issues can be sorted out.

Question: how hard would it be to backport the work you've done to Python 3.2?
Obviously I'm not saying it should be ported to the official 3.2 branch, but
if *someone* were interested in doing so, would it be possible?  Sounds like
you can almost get there with stdlib changes, but would require a few C
changes too (I haven't looked at the diff yet).  I'm just wondering if the
same API could be made available to Python 3.2 as a third party module.  It
sounds like "almost, but not quite".

>If people want to experiment with this code without cloning and building, I
>created a Debian package using checkinstall, which can be installed using
>
>sudo dpkg -i pythonv_3.3-1_i386.deb
>
>and removed using
>
>sudo dpkg -r pythonv
>
>I can make this Debian package available for download, if anyone wants it.

Is the Debian packaging branch available too?  I'd be happy to throw that in
my PPA for Ubuntu users to play with.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110613/cb50037f/attachment.pgp>

From vinay_sajip at yahoo.co.uk  Mon Jun 13 18:00:30 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Jun 2011 16:00:30 +0000 (UTC)
Subject: [Python-Dev] In-Python virtualisation and packaging
References: <loom.20110613T132530-565@post.gmane.org>
	<20110613110224.05d6962e@neurotica.wooz.org>
Message-ID: <loom.20110613T174853-57@post.gmane.org>

Barry Warsaw <barry <at> python.org> writes:

> Question: how hard would it be to backport the work you've done to Python 3.2?
> Obviously I'm not saying it should be ported to the official 3.2 branch, but
> if *someone* were interested in doing so, would it be possible?  Sounds like
> you can almost get there with stdlib changes, but would require a few C
> changes too (I haven't looked at the diff yet).  I'm just wondering if the
> same API could be made available to Python 3.2 as a third party module.  It
> sounds like "almost, but not quite".

I think it's feasible - as far as I know, there's nothing 3.3 specific about the
changes that were made, other than just happening to be against the default
branch. AFAIK the getpath.c/getpathp.c changes will also work on 3.2, as all
they do is look for a config file in a specific place and read a path from it if
it's there. If it's not there, no biggie. If it's there, it sets up the
sys.prefix/sys.exec_prefix values from that path.

Possibly Carl's original Python changes would be easier to work from, since the
sysconfig stuff has now changed quite a bit because of packaging coming in to
cpython. For one thing, the _INSTALL_SCHEMES dict is replaced by reading that
data from a config file.

> Is the Debian packaging branch available too?  I'd be happy to throw that in
> my PPA for Ubuntu users to play with.

My Debian-packaging-fu is not that good, I'm afraid, so there's no branch for
the .deb, as such. I made the package by running make and then

sudo checkinstall -D --fstrans=no

which takes forever (God knows why - it's many many minutes at 100% CPU) but
eventually comes up with the .deb.

Regards,

Vinay Sajip


From dmalcolm at redhat.com  Mon Jun 13 18:21:19 2011
From: dmalcolm at redhat.com (David Malcolm)
Date: Mon, 13 Jun 2011 12:21:19 -0400
Subject: [Python-Dev] is anyone using Misc/RPM?
In-Reply-To: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>
References: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>
Message-ID: <1307982079.16924.5309.camel@surprise>

On Sun, 2011-06-12 at 15:37 -0500, Benjamin Peterson wrote:
> If no one is using it, I'd like to delete it. I also don't think we
> should be in business of distributing distribution specific files.

FWIW, Fedora and RHEL don't use this particular .spec file; we roll our
own.

I can't speak for all of the other RPM-using distributions, of course.


From vinay_sajip at yahoo.co.uk  Mon Jun 13 18:25:11 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Jun 2011 16:25:11 +0000 (UTC)
Subject: [Python-Dev] Implement Aspect-oriented programming
References: <BANLkTin46+QbM3ScxC=vw0z-GpU_ZqcvDw@mail.gmail.com>
Message-ID: <loom.20110613T182153-569@post.gmane.org>

Jiawei Li <jiawei.h.li <at> gmail.com> writes:

> For example, the logging module is not very useful right now, as it requires
sprinkling small one-liners all over my code - not exactly ideal.
> Why not take a page from aspect-oriented programming and allow for injection
of code with point cuts?

If you're only interested in logging method entry and exit - in which case,
you're not really using logging to its full potential - then an AOP style
approach may work for you. But the point of logging is to send messages to
yourself (and others) from your code, and an AOP approach will not lend itself
to intelligent, context-sensitive messages.

Regards,

Vinay Sajip





From barry at python.org  Mon Jun 13 18:41:43 2011
From: barry at python.org (Barry Warsaw)
Date: Mon, 13 Jun 2011 12:41:43 -0400
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <loom.20110613T174853-57@post.gmane.org>
References: <loom.20110613T132530-565@post.gmane.org>
	<20110613110224.05d6962e@neurotica.wooz.org>
	<loom.20110613T174853-57@post.gmane.org>
Message-ID: <20110613124143.1ca85583@neurotica.wooz.org>

On Jun 13, 2011, at 04:00 PM, Vinay Sajip wrote:

>My Debian-packaging-fu is not that good, I'm afraid, so there's no branch for
>the .deb, as such. I made the package by running make and then
>
>sudo checkinstall -D --fstrans=no
>
>which takes forever (God knows why - it's many many minutes at 100% CPU) but
>eventually comes up with the .deb.

Ah, no I don't think that'll be helpful.  I can probably reuse the python3.3
packaging stuff to do a PPA.  (It takes that long because it basically does a
`make install`.)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110613/d41c06fc/attachment.pgp>

From opensourcesurvey at cs.ua.edu  Mon Jun 13 18:45:55 2011
From: opensourcesurvey at cs.ua.edu (Jeffrey Carver)
Date: Mon, 13 Jun 2011 11:45:55 -0500
Subject: [Python-Dev] Participation Requested: Survey about Open-Source
	Software Development
Message-ID: <00c601cc29e9$5d706eb0$18514c10$@cs.ua.edu>

Hi,

Drs. Jeffrey Carver, Rosanna Guadagno, Debra McCallum, and Mr. Amiangshu
Bosu,  University of Alabama, and Dr. Lorin Hochstein, University of
Southern California, are conducting a survey of open-source software
developers. This survey seeks to understand how developers on distributed,
virtual teams, like open-source projects, interact with each other to
accomplish their tasks. You must be at least 19 years of age to complete the
survey. The survey should take approximately 15 minutes to complete.

If you are actively participating as a developer, please consider completing
our survey.
 
Here is the link to the survey:   http://goo.gl/HQnux

We apologize for inconvenience and if you receive multiple copies of this
email. This survey has been approved by The University of Alabama IRB board.

Thanks,

Dr. Jeffrey Carver
Assistant Professor
University of Alabama
(v) 205-348-9829  (f) 205-348-0219
http://www.cs.ua.edu/~carver



From vinay_sajip at yahoo.co.uk  Mon Jun 13 19:31:27 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 13 Jun 2011 17:31:27 +0000 (UTC)
Subject: [Python-Dev] In-Python virtualisation and packaging
References: <loom.20110613T132530-565@post.gmane.org>
	<20110613110224.05d6962e@neurotica.wooz.org>
	<loom.20110613T174853-57@post.gmane.org>
	<20110613124143.1ca85583@neurotica.wooz.org>
Message-ID: <loom.20110613T192403-67@post.gmane.org>

Barry Warsaw <barry <at> python.org> writes:

> Ah, no I don't think that'll be helpful.  I can probably reuse the python3.3
> packaging stuff to do a PPA.

Okay, go for it! Is there a specific tutorial somewhere about making a PPA for
Python? (As opposed to more generalised tutorials - or would they be sufficient?)

> (It takes that long because it basically does a
> `make install`.)

I realise that, as well as recording what it's doing, but that part seems to
happen fairly quickly. 

Then it says "Copying files to the temporary directory..." and that part seems
to take forever. The whole deb is under 25MB, what could be taking many minutes?

Regards,

Vinay


From martin at v.loewis.de  Mon Jun 13 21:03:18 2011
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 13 Jun 2011 21:03:18 +0200
Subject: [Python-Dev] is anyone using Misc/RPM?
In-Reply-To: <BANLkTi=XYh9FT4jqZK5D2Q-b5cLQ2JPsuQ@mail.gmail.com>
References: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>	<4DF5321B.4000603@v.loewis.de>
	<BANLkTi=XYh9FT4jqZK5D2Q-b5cLQ2JPsuQ@mail.gmail.com>
Message-ID: <4DF65EF6.6020507@v.loewis.de>

>> I disagree. We certainly include PCbuild/*.vcproj, and Tools/msi,
>> which are also "distribution-specific". Likewise, we have plenty
>> of OSX-specific files (including special-casing for specific releases
>> thereof).
> 
> I should qualify: We shouldn't distribute distribution-specific files
> for which we don't provide binaries.

Hmm. We have VS6, VS7, and VS8 project files, OS/2 makefiles, and
configure.in has specifics for Solaris, even though we don't provide
binaries for any of these. I don't think that's a useful principle
to follow.

Regards,
Martin

From martin at v.loewis.de  Mon Jun 13 21:36:24 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 13 Jun 2011 21:36:24 +0200
Subject: [Python-Dev] is anyone using Misc/RPM?
In-Reply-To: <1307982079.16924.5309.camel@surprise>
References: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>
	<1307982079.16924.5309.camel@surprise>
Message-ID: <4DF666B8.7060205@v.loewis.de>

Am 13.06.2011 18:21, schrieb David Malcolm:
> On Sun, 2011-06-12 at 15:37 -0500, Benjamin Peterson wrote:
>> If no one is using it, I'd like to delete it. I also don't think we
>> should be in business of distributing distribution specific files.
> 
> FWIW, Fedora and RHEL don't use this particular .spec file; we roll our
> own.
> 
> I can't speak for all of the other RPM-using distributions, of course.

I doubt any of the distributions uses it, but that's not it's purpose,
either. Instead, it is meant for people who want to roll their own
RPM.

Regards,
Martin

From wolfson at gmail.com  Mon Jun 13 22:00:36 2011
From: wolfson at gmail.com (Ben Wolfson)
Date: Mon, 13 Jun 2011 13:00:36 -0700
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
In-Reply-To: <4DF3FA53.7000906@canterbury.ac.nz>
References: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
	<BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>
	<BANLkTino_LOtR=UcTT4UfV=kGtUiaKNG_A@mail.gmail.com>
	<4DF3FA53.7000906@canterbury.ac.nz>
Message-ID: <BANLkTik_=t74e5h2ntfu9BaLq02n5kaYXg@mail.gmail.com>

On Sat, Jun 11, 2011 at 4:29 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Ben Wolfson wrote:
>>
>> You can't have an internal replacement
>> field in this part of the replacement field, so '{' can always safely
>> be assumed to be Just a Brace and not the start of a replacement
>> field, regardless of whether it's doubled,
>
> I'm worried that the rules in this area are getting too
> complicated for a human to follow. If braces are allowed
> as plain data between square brackets and/or vice versa,
> it's going to be a confusing mess to read, and there will
> always be some doubt in the programmer's mind as to whether
> they have to be escaped somehow or not.
>
> I'm inclined to think that any such difficult cases should
> simply be disallowed. If the docs say an identifier is required
> someplace, the implementation should adhere strictly to that.

There are two cases with the braces: attribute selection and item
selection. The docs say that attributes should be identifiers, and
that the argument name should be an integer or an identifier, but that
the item selector can essentially be an arbitrary string as long as it
doesn't contain ']', which indicates its end. The docs as they stand
suggest that braces in item selectors should be treated as plain data:

"""
Format strings contain ?replacement fields? surrounded by curly braces
{}. Anything that is not contained in braces is considered literal
text, which is copied unchanged to the output. If you need to include
a brace character in the literal text, it can be escaped by doubling:
{{ and }}.
"""

Since it mentions escaping only in the context of how to get a brace
in literal text rather than in a replacement field.

Current behavior is to perform escapes in the format spec part of a
replacement field, and that, I think, makes sense, since there can be
an internal replacement field there. However, it's still pretty simple
to tell whether braces need to be escaped or not: a brace in the
format spec does need to be escaped, a brace before the format spec
doesn't.

> It's not *that* hard to parse an indentifier properly, and
> IMO any use case that requires putting arbitrary characters
> into an item selector is abusing the format mechanism and
> should be redesigned to work some other way.

If by "item selector" you mean (using the names from the grammar in
the docs) the element_index, I don't see why this should be the case;
dictionaries can contain non-identified keys, after all. If you mean
the attribute_name (and arg_name) parts, then requiring an identifier
(or an integer for arg_name) makes a lot more sense.

I assume that Talin had some reason for stating otherwise in the PEP
(one of the few things that does get explicitly said about the
field_name part), but I'm kind of at a loss for why; you would need to
have a custom __getattribute__ to exploit it, and it would be a lot
less confusing just to use __getitem__.

-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]

From solipsis at pitrou.net  Mon Jun 13 22:32:00 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 13 Jun 2011 22:32:00 +0200
Subject: [Python-Dev] is anyone using Misc/RPM?
References: <BANLkTinDbb244_37D_TrLrH7e+-ZyA5xYA@mail.gmail.com>
	<4DF5321B.4000603@v.loewis.de>
	<BANLkTi=XYh9FT4jqZK5D2Q-b5cLQ2JPsuQ@mail.gmail.com>
	<4DF65EF6.6020507@v.loewis.de>
Message-ID: <20110613223200.3039a8d1@pitrou.net>

On Mon, 13 Jun 2011 21:03:18 +0200
"Martin v. L?wis" <martin at v.loewis.de> wrote:

> >> I disagree. We certainly include PCbuild/*.vcproj, and Tools/msi,
> >> which are also "distribution-specific". Likewise, we have plenty
> >> of OSX-specific files (including special-casing for specific releases
> >> thereof).
> > 
> > I should qualify: We shouldn't distribute distribution-specific files
> > for which we don't provide binaries.
> 
> Hmm. We have VS6, VS7, and VS8 project files, OS/2 makefiles, and
> configure.in has specifics for Solaris, even though we don't provide
> binaries for any of these. I don't think that's a useful principle
> to follow.

Well, if we want to nitpick, all these files are for compilation, no
for distribution ;)

Regards

Antoine.



From greg.ewing at canterbury.ac.nz  Mon Jun 13 23:54:17 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 14 Jun 2011 09:54:17 +1200
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <E784DF2D-A8F9-4D81-998E-77F2905C9AB2@gmail.com>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
	<6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
	<4DF53B4D.1000607@canterbury.ac.nz>
	<E784DF2D-A8F9-4D81-998E-77F2905C9AB2@gmail.com>
Message-ID: <4DF68709.8070206@canterbury.ac.nz>

Raymond Hettinger wrote:

> How would you describe the creation of a lazy result
> that keeps a reference to the underlying buffer?

I'd call it a view. There is plenty of precedence for this
kind of object in Python -- I gave a few examples before.

The distinguishing feature of ropes, as I understand the
term, is that you get a lazy object when you *combine* other
objects, e.g. concatenating strings.

That's not being asked for here -- there is only taking
apart going on, not putting together.

It's also different from the oft-rejected idea of lazy
string slicing, because extracting a field would give
you a separate object, e.g. an int or string, not a
reference to the original data object being unpacked.

So I really can't see what harm it could do, except for
maybe a tiny performance reduction in the case where you
extract all the fields, or refer to extracted fields
repeatedly.

Even if that turned out to be the case, and was considered
objectionable, it could still be useful to provide
view-oriented unpacking as an option.

-- 
Greg

From ethan at stoneleaf.us  Tue Jun 14 00:11:15 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 13 Jun 2011 15:11:15 -0700
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <4DF4F4E0.5060003@stoneleaf.us>
References: <4DD2C2A5.3080403@stoneleaf.us>	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>	<4DD2D89D.4000303@stoneleaf.us>	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>	<4DD2F661.2050005@stoneleaf.us>	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us>
Message-ID: <4DF68B03.2010603@stoneleaf.us>

Thank you all for the responses.  Rather than reply to each, I just made 
one big summary.  :)
----------------------------------------------------------------

Martin v. L?wis wrote:
 > Ethan Furman wrote:
 >> # constants
 >>
 >> EOH  = b'\r'[0]
 >> CHAR = b'C'[0]
 >> DATE = b'D'[0]
 >> FLOAT = b'F'[0]
 >> INT = b'I'[0]
 >> LOGICAL = b'L'[0]
 >> MEMO = b'M'[0]
 >> NUMBER = b'N'[0]
 >>
 >> This is not beautiful code.
 >
 > In this case, I think the intent would be better captured with
 >
 > def ASCII(c):
 >     return c.encode('ascii')
 >
 > EOH     = ASCII('\r') # 0D
 > CHAR    = ASCII('C')  # 43
 > DATE    = ASCII('D')  # 44
 > FLOAT   = ASCII('F')  # 46
 > INT     = ASCII('I')  # 49
 > LOGICAL = ASCII('L')  # 4C
 > MEMO    = ASCII('M')  # 4D
 > NUMBER  = ASCII('N')  # 4E
 >
 > This expresses the intent that a) these are really byte values,
 > not characters, and b) the specific choice of byte values was
 > motivated by ASCII.

Definitely easier to read.  If I go this route I'll probably use ord(), 
though, since ascii and unicode are the same for the first 127 chars, 
and there will be plenty of places to error out with a more appropriate 
message if I get garbage.  Since I really don't care what the actual 
integer values are, I'll skip those comments, too.
----------------------------------------------------------------

Hagen F?rstenau wrote:
 > You still have the alternative
 >
 > EOH = ord('\r')
 > CHAR = ord('C')
 > ...
 >
 > which looks fine to me.

Yes it does.  I just dislike the (to me unnecessary) extra function 
call.  For those tuning in late to this thread, these are workarounds 
for this not working:

field_type = header[11] # field_type is now an int, not a 1-byte bstr
if field_type == r'C':  # r'C' is a 1-byte bstr, so this always fails
----------------------------------------------------------------

Greg Ewing wrote:
 > Guido van Rossum wrote:
 >>> On Thu, May 19, 2011 at 1:43 AM, Nick Coghlan wrote:
 >>>> Proposals to address this include:
 >>>> - introduce a "character" literal to allow c'a' as an alternative
 >>>> to ord('a')
 >>
 >> -1; the result is not a *character* but an integer.
 >
 > Would you be happier if it were spelled i'a' instead?

That would work for me, although I would prefer a'a' (for ASCII).  :)
----------------------------------------------------------------

Stephen J. Turnbull wrote:
 > Put mascara on a pig, and you have a pig with mascara on, not Bette
 > Davis.  I don't necessarily think you're doing anybody a service by
 > making the hack of using ASCII bytes as mnemonics more beautiful.  I
 > think Martin's version is as beautiful as this code should get.

I'll either use Martin's or Nick's.  The point of beauty here is the 
ease of readability.  I think less readable is worse, and we shouldn't 
have to have ugly, hard to read code nor inefficient code just because 
we have to deal with byte streams that aren't unicode.
----------------------------------------------------------------

Nick Coghlan wrote:
 > Agreed, but:
 >
 > EOH, CHAR, DATE, FLOAT, INT, LOGICAL, MEMO, NUMBER = b'\rCDFILMN'
 >
 > is a shorter way to write the same thing.
 >
 > Going two per line makes it easier to mentally map the characters:
 >
 > EOH, CHAR = b'\rC'
 > DATE, FLOAT = b'DF'
 > INT, LOGICAL = b'IL'
 > MEMO, NUMBER = b'MN'

Wow.  I didn't realize that could be done.  That very nearly makes up 
for not being able to do it one char at a time.

Thanks, Nick!
----------------------------------------------------------------

~Ethan~

From carl at oddbird.net  Tue Jun 14 01:27:05 2011
From: carl at oddbird.net (Carl Meyer)
Date: Mon, 13 Jun 2011 18:27:05 -0500
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <4DF5FAC8.4090606@voidspace.org.uk>
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
Message-ID: <4DF69CC9.9000704@oddbird.net>



On 06/13/2011 06:55 AM, Michael Foord wrote:
> There are two options:
>
> Bring the full functionality into the standard library so that Python
> supports virtual environments out of the box. As is the case with adding
> anything to the standard library it will then be impossible to add
> features to the virtualization support in Python 3.3 once 3.3 is
> released - new features will go into 3.4.

I think it's not hard to provide enough hooks to allow third-party tools
to extend the virtualenv-creation process, while still having enough
code in the stdlib to allow actual creation of virtualenvs. Virtualenv
already has very few features, and doesn't get very much by way of new
feature requests -- all the UI sugar and significant shell integration
goes into Doug Hellmann's virtualenvwrapper, and I wouldn't foresee that
changing.

IOW, I don't think the maintenance concerns outweigh the benefits of
being able to create virtualenvs with an out-of-the-box Python.

> Add only the minimal changes required to support a third-party virtual
> environment tool.
> 
> Virtual environments are phenomenally useful, so I would support having
> the full tool in the standard library, but it does raise maintenance and
> development issues.
> 
> Don't forget windows support! ;-)
> 
> All the best,
> 
> Michael Foord

From carl at oddbird.net  Tue Jun 14 01:46:33 2011
From: carl at oddbird.net (Carl Meyer)
Date: Mon, 13 Jun 2011 18:46:33 -0500
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <BANLkTikqG-VWkaSTGH3bb5JACcVZVcOR_A@mail.gmail.com>
References: <loom.20110613T132530-565@post.gmane.org>	<4DF5FAC8.4090606@voidspace.org.uk>	<BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>	<loom.20110613T142559-463@post.gmane.org>
	<BANLkTikqG-VWkaSTGH3bb5JACcVZVcOR_A@mail.gmail.com>
Message-ID: <4DF6A159.1010105@oddbird.net>

On 06/13/2011 08:07 AM, Nick Coghlan wrote:
> On Mon, Jun 13, 2011 at 10:50 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> You're right in terms of the current Python ecosystem and 3.x adoption, because
>> of course this approach requires support from Python itself in terms of its
>> site.py code. However, virtual environments have a utility beyond supporting
>> older Pythons on newer OSes, since another common use case is having different
>> library environments sandboxed from each other on different projects, even if
>> all those projects are using Python 3.3+.
> 
> Yeah, even if the innate one struggles on later OS releases that
> changed things in a backwards incompatible way, it will still be
> valuable on the OS versions that are around at the time that version
> of Python gets released.

FWIW, historically pretty much every new Python version has broken
virtualenv, and new OS versions almost never have. Virtualenv isn't
especially OS-dependent (not nearly as much as some other stdlib
modules): the most OS-dependent piece is "shell activation", and that's
a feature I would prefer to entirely leave out of the stdlib virtualenv
(it's a convenience, not a necessity for virtualenv use, and the need to
maintain it for a variety of OS shells is a maintenance burden I don't
think Python should adopt).

In fact, the only new-OS-version adjustment I can recall virtualenv
needing to make is when Debian introduced dist-packages -- but even that
doesn't really apply, as that was distro-packager change to Python
itself. With a built-in virtualenv it would be the distro packagers
responsibility to make sure their patch to Python doesn't break the
virtualenv module.

So I don't think a virtualenv stdlib module would be at all likely to
break on a new OS release, if Python itself is not broken by that OS
release. (It certainly wouldn't be the stdlib module most likely to be
broken by OS changes, in comparison to e.g. shutil, threading...)

Carl

From pje at telecommunity.com  Tue Jun 14 01:40:29 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 13 Jun 2011 19:40:29 -0400
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <4DF68B03.2010603@stoneleaf.us>
References: <4DD2C2A5.3080403@stoneleaf.us>
	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>
	<4DD2D89D.4000303@stoneleaf.us>
	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>
	<4DD2F661.2050005@stoneleaf.us>
	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>
	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us> <4DF68B03.2010603@stoneleaf.us>
Message-ID: <20110613234035.8D9A33A40D9@sparrow.telecommunity.com>

At 03:11 PM 6/13/2011 -0700, Ethan Furman wrote:
>Nick Coghlan wrote:
> > Agreed, but:
> >
> > EOH, CHAR, DATE, FLOAT, INT, LOGICAL, MEMO, NUMBER = b'\rCDFILMN'
> >
> > is a shorter way to write the same thing.
> >
> > Going two per line makes it easier to mentally map the characters:
> >
> > EOH, CHAR = b'\rC'
> > DATE, FLOAT = b'DF'
> > INT, LOGICAL = b'IL'
> > MEMO, NUMBER = b'MN'
>
>Wow.  I didn't realize that could be done.  That very nearly makes 
>up for not being able to do it one char at a time.

You can still do it one at a time:

CHAR, = b'C'
INT,  = b'I'
...

etc.  I just tried it with Python 3.1 and it works there.


From carl at oddbird.net  Tue Jun 14 01:50:44 2011
From: carl at oddbird.net (Carl Meyer)
Date: Mon, 13 Jun 2011 18:50:44 -0500
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <4DF6A159.1010105@oddbird.net>
References: <loom.20110613T132530-565@post.gmane.org>	<4DF5FAC8.4090606@voidspace.org.uk>	<BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>	<loom.20110613T142559-463@post.gmane.org>	<BANLkTikqG-VWkaSTGH3bb5JACcVZVcOR_A@mail.gmail.com>
	<4DF6A159.1010105@oddbird.net>
Message-ID: <4DF6A254.8030405@oddbird.net>

On 06/13/2011 06:46 PM, Carl Meyer wrote:
> FWIW, historically pretty much every new Python version has broken
> virtualenv

I should clarify that this is because of the delicate stdlib
bootstrapping virtualenv currently has to do; the built-in virtualenv
eliminates this entirely and will require much less maintenance for new
Python versions.

Carl

From fuzzyman at voidspace.org.uk  Tue Jun 14 02:00:45 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 14 Jun 2011 01:00:45 +0100
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <4DF6A159.1010105@oddbird.net>
References: <loom.20110613T132530-565@post.gmane.org>	<4DF5FAC8.4090606@voidspace.org.uk>	<BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>	<loom.20110613T142559-463@post.gmane.org>	<BANLkTikqG-VWkaSTGH3bb5JACcVZVcOR_A@mail.gmail.com>
	<4DF6A159.1010105@oddbird.net>
Message-ID: <4DF6A4AD.1060706@voidspace.org.uk>

On 14/06/2011 00:46, Carl Meyer wrote:
> [snip...]
> So I don't think a virtualenv stdlib module would be at all likely to
> break on a new OS release, if Python itself is not broken by that OS
> release. (It certainly wouldn't be the stdlib module most likely to be
> broken by OS changes, in comparison to e.g. shutil, threading...)
>

And if we gain Carl as a Python committer to help maintain it, then I'd 
say it is worth doing for that reason alone...

Michael

> Carl
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From ncoghlan at gmail.com  Tue Jun 14 02:35:41 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 14 Jun 2011 10:35:41 +1000
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <20110613234035.8D9A33A40D9@sparrow.telecommunity.com>
References: <4DD2C2A5.3080403@stoneleaf.us>
	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>
	<4DD2D89D.4000303@stoneleaf.us>
	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>
	<4DD2F661.2050005@stoneleaf.us>
	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>
	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us> <4DF68B03.2010603@stoneleaf.us>
	<20110613234035.8D9A33A40D9@sparrow.telecommunity.com>
Message-ID: <BANLkTinBS7kQjU-eEG7PoD=eMeRPx3PV+w@mail.gmail.com>

On Tue, Jun 14, 2011 at 9:40 AM, P.J. Eby <pje at telecommunity.com> wrote:
> You can still do it one at a time:
>
> CHAR, = b'C'
> INT, ?= b'I'
> ...
>
> etc. ?I just tried it with Python 3.1 and it works there.

I almost mentioned that, although it does violate one of the
"unwritten rules of the Zen" (in this case, "syntax shall not look
like grit on Tim's monitor")

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From greg.ewing at canterbury.ac.nz  Tue Jun 14 02:36:16 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 14 Jun 2011 12:36:16 +1200
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
In-Reply-To: <BANLkTik_=t74e5h2ntfu9BaLq02n5kaYXg@mail.gmail.com>
References: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
	<BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>
	<BANLkTino_LOtR=UcTT4UfV=kGtUiaKNG_A@mail.gmail.com>
	<4DF3FA53.7000906@canterbury.ac.nz>
	<BANLkTik_=t74e5h2ntfu9BaLq02n5kaYXg@mail.gmail.com>
Message-ID: <4DF6AD00.3000207@canterbury.ac.nz>

Ben Wolfson wrote:

> If by "item selector" you mean (using the names from the grammar in
> the docs) the element_index, I don't see why this should be the case;
> dictionaries can contain non-identified keys, after all.

Of course they can, but that's not the point. The point is
that putting arbitrary strings between [...] in a format
spec without any form of quoting or requirement for bracket
matching leads to something that's too confusing for humans
to read.

IMO the spec should be designed so that the format string
can be parsed using the same lexical analysis rules as
Python code. That means anything that is meant to "hang
together" as a single unit, such as an item selector,
needs to look like a single Python token, e.g. an integer
or identifier.

I realise this is probably more restrictive than the PEP
suggests, but I think it would be better that way all round.

-- 
Greg

From cs at zip.com.au  Tue Jun 14 03:35:03 2011
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 14 Jun 2011 11:35:03 +1000
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <loom.20110613T154326-403@post.gmane.org>
References: <loom.20110613T154326-403@post.gmane.org>
Message-ID: <20110614013503.GA29499@cskk.homeip.net>

On 13Jun2011 13:47, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
| Nick Coghlan <ncoghlan <at> gmail.com> writes:
| 
| > On Mon, Jun 13, 2011 at 10:57 PM, Antoine Pitrou <solipsis <at> pitrou.net>
| > wrote:
| > > As a side-note, I think calling it "virtualization" is a recipe for
| > > confusion.
| > 
| > Indeed, OS level virtualisation pretty much has a lock on that term.
| > "virtual environments" skates close to it but manages to avoid it well
| > enough to avoid confusion.
| 
| Or as they involving encapsulating paths and libaries, perhaps we could call
| them "capsules" ;-) though I think the term virtualenv is pretty entrenched now
| in the Python community.

"virtualenv" by all means - we all know what is meant.

But "virtualisation" - I also am -1 on that. Indeed, when I started reading
this thread my expectation was wrong for that very reason.

Same issue with "capsules" (yes I know you weren't serious) - too
generic a term, too vague.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

It looked good-natured, she thought;  Still it had very long claws and a
great many teeth, so she felt it ought to be treated with respect.

From cs at zip.com.au  Tue Jun 14 03:43:58 2011
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 14 Jun 2011 11:43:58 +1000
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <loom.20110613T192403-67@post.gmane.org>
References: <loom.20110613T192403-67@post.gmane.org>
Message-ID: <20110614014358.GA30397@cskk.homeip.net>

On 13Jun2011 17:31, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
| Barry Warsaw <barry <at> python.org> writes:
| > Ah, no I don't think that'll be helpful.  I can probably reuse the python3.3
| > packaging stuff to do a PPA.
| 
| Okay, go for it! Is there a specific tutorial somewhere about making a PPA for
| Python? (As opposed to more generalised tutorials - or would they be sufficient?)
| 
| > (It takes that long because it basically does a
| > `make install`.)
| 
| I realise that, as well as recording what it's doing, but that part seems to
| happen fairly quickly. 
| 
| Then it says "Copying files to the temporary directory..." and that part seems
| to take forever. The whole deb is under 25MB, what could be taking many minutes?

[ wild speculation ... ]

How does it decide what to copy? If it is a "blind" make-me-a-package
tool it may be scanning the whole OS install or something (expensive
but linear) and maybe then doing some ghastly O(n^2) changed file
comparison. Inefficient comparison stuff leaks into the real world all
the time; the Linux kernel installs have a "hardlinks" program which
is one of my pet hates for this very reason - it runs over the modules
trees looking for identical module files to hard link and if you've got
several kernels lying around it is unforgivably slow. Or maybe it loads
the package install db into memory and does something expensive to see
what's not accounted for.

[ end speculation, but nothing useful now follows ... ]

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

"He deserves death!"
"Deserves it!  I daresay he does.  And many die that deserve life.  Is it in
 your power to give it to them?  Then do not be so quick to deal out death in
 judgement, for even the very wise may not see all ends."
        - Gandalf, _The Lord of the Rings_

From wolfson at gmail.com  Tue Jun 14 03:46:51 2011
From: wolfson at gmail.com (Ben Wolfson)
Date: Mon, 13 Jun 2011 18:46:51 -0700
Subject: [Python-Dev] PEP 3101 implementation vs. documentation
In-Reply-To: <4DF6AD00.3000207@canterbury.ac.nz>
References: <BANLkTikP72XV2UmnN2doJ2uDbvXcM_qFXQ@mail.gmail.com>
	<BANLkTimig0Yx2h7u8maVoEcp3RgURV=U=w@mail.gmail.com>
	<BANLkTino_LOtR=UcTT4UfV=kGtUiaKNG_A@mail.gmail.com>
	<4DF3FA53.7000906@canterbury.ac.nz>
	<BANLkTik_=t74e5h2ntfu9BaLq02n5kaYXg@mail.gmail.com>
	<4DF6AD00.3000207@canterbury.ac.nz>
Message-ID: <BANLkTikD6No7FVBa8sP06D8e0S8f8fW5ig@mail.gmail.com>

On Mon, Jun 13, 2011 at 5:36 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Ben Wolfson wrote:
>
>> If by "item selector" you mean (using the names from the grammar in
>> the docs) the element_index, I don't see why this should be the case;
>> dictionaries can contain non-identified keys, after all.
>
> Of course they can, but that's not the point. The point is
> that putting arbitrary strings between [...] in a format
> spec without any form of quoting or requirement for bracket
> matching leads to something that's too confusing for humans
> to read.

But there is a requirement for bracket matching: the "[" that opens
the element_index is matched by the next "]". Arguably (as Terry Reedy
said) this is also a form of quoting, in which the square brackets are
the quotation operators. It seems no more confusing to me than
allowing arbitrary strings between in '"..."'; those quotation marks
aren't even oriented. (Admittedly, syntax highlighting helps there.)

Compared to this: "{0: ^+#10o}", a string like this: "this is normal
text, but {e.underline[this text is is udnerlined {sic}!]}---and we're
back to normal now" is pretty damn readable to this human, nor do I
see what about the rule "when you see a [, keep going until you see a
]" is supposed to be insuperably confusing. (Compare---not that it
helps my case in regard to readability---grouping in regular
expressions, where you don't usually have the aid of special syntax
highlighting inside the string; you see a '(', you know that you've
encountered a group which continues until the next (unescaped!) ')'.
The stuff that comes inside the parentheses might look like line
noise---and the whole thing might look like line noise---but *that*
rule about the structure of a regexp is pretty straightforward.)

> IMO the spec should be designed so that the format string
> can be parsed using the same lexical analysis rules as
> Python code. That means anything that is meant to "hang
> together" as a single unit, such as an item selector,
> needs to look like a single Python token, e.g. an integer
> or identifier.

If that's the rationale, why not change the spec so that instead of this:

"{0[spam]}"

You do this:

"{0['spam']}"

? Hangs together; single Python token. Bonus: it would make it
possible for this to work:

(a) "{0['12']}".format({'12': 4})

whereas currently this:

"{0[12]}".format(...)

passes the integer 12 to __getitem__, and (a) passes the string "'12'".

(Discovery: the "abuse" of the format mechanism I want to perpetrate
via element_index can also be perpetrated with a custom __format__
method:

>>> class foo:
...        def __format__(self, a): return a * 2
...
>>> "hello {0::![} world".format(foo())
'hello :![:![ world'

. So any reform to make it impossible to use str.format creatively
will have to be fairly radical. I actually think that my intended
abuse is actually a perfectly reasonable use, but it would be
disallowed if only integers and identifiers can be in the
element_index field.)

-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]

From barry at python.org  Tue Jun 14 04:01:26 2011
From: barry at python.org (Barry Warsaw)
Date: Mon, 13 Jun 2011 22:01:26 -0400
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <4DF6A4AD.1060706@voidspace.org.uk>
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
	<BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>
	<loom.20110613T142559-463@post.gmane.org>
	<BANLkTikqG-VWkaSTGH3bb5JACcVZVcOR_A@mail.gmail.com>
	<4DF6A159.1010105@oddbird.net> <4DF6A4AD.1060706@voidspace.org.uk>
Message-ID: <20110613220126.73a36b39@neurotica.wooz.org>

On Jun 14, 2011, at 01:00 AM, Michael Foord wrote:

>On 14/06/2011 00:46, Carl Meyer wrote:
>> [snip...]
>> So I don't think a virtualenv stdlib module would be at all likely to
>> break on a new OS release, if Python itself is not broken by that OS
>> release. (It certainly wouldn't be the stdlib module most likely to be
>> broken by OS changes, in comparison to e.g. shutil, threading...)
>>
>
>And if we gain Carl as a Python committer to help maintain it, then I'd say
>it is worth doing for that reason alone...

+1
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110613/73e33930/attachment.pgp>

From exarkun at twistedmatrix.com  Tue Jun 14 03:56:06 2011
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Tue, 14 Jun 2011 01:56:06 -0000
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <BANLkTinBS7kQjU-eEG7PoD=eMeRPx3PV+w@mail.gmail.com>
References: <4DD2C2A5.3080403@stoneleaf.us>
	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>
	<4DD2D89D.4000303@stoneleaf.us>
	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>
	<4DD2F661.2050005@stoneleaf.us>
	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>
	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us> <4DF68B03.2010603@stoneleaf.us>
	<20110613234035.8D9A33A40D9@sparrow.telecommunity.com>
	<BANLkTinBS7kQjU-eEG7PoD=eMeRPx3PV+w@mail.gmail.com>
Message-ID: <20110614015606.1894.1336593159.divmod.xquotient.147@localhost.localdomain>

On 12:35 am, ncoghlan at gmail.com wrote:
>On Tue, Jun 14, 2011 at 9:40 AM, P.J. Eby <pje at telecommunity.com> 
>wrote:
>>You can still do it one at a time:
>>
>>CHAR, = b'C'
>>INT, ?= b'I'
>>...
>>
>>etc. ?I just tried it with Python 3.1 and it works there.
>
>I almost mentioned that, although it does violate one of the
>"unwritten rules of the Zen" (in this case, "syntax shall not look
>like grit on Tim's monitor")

    [CHAR] = b'C'
    [INT]  = b'I'
    ...

Jean-Paul

From larry at hastings.org  Tue Jun 14 04:36:05 2011
From: larry at hastings.org (Larry Hastings)
Date: Mon, 13 Jun 2011 19:36:05 -0700
Subject: [Python-Dev] Call For Topics for Python Language Summit at
	EuroPython 2011
Message-ID: <4DF6C915.50803@hastings.org>



Howdy howdy.  Michael Foord can't make it to EuroPython this year--and 
congratulations to him on his new baby!--so he's asked me to chair the 
Python Language Summit in his stead.  Although I have some suggested 
topics from Michael, I don't have any burning issues of my own to add to 
the docket.  That's where *you* come in.

If you have a topic you'd like to suggest, please email it privately to 
me (larry at hastings dot org) along with an estimate of how long you 
think it'll take to discuss.  Please have all suggestions to me no later 
than Friday June 17th.  Also, please include the phrase "Python Language 
Summit" in the subject line, in case it gets caught in my spam filters.  
You don't have to be attending the summit to suggest a topic--I'll 
consider suggestions from anybody.

Hope to see you at EuroPython,


/larry/

From jannis at leidel.info  Tue Jun 14 11:15:20 2011
From: jannis at leidel.info (Jannis Leidel)
Date: Tue, 14 Jun 2011 11:15:20 +0200
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <4DF6A159.1010105@oddbird.net>
References: <loom.20110613T132530-565@post.gmane.org>	<4DF5FAC8.4090606@voidspace.org.uk>	<BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>	<loom.20110613T142559-463@post.gmane.org>
	<BANLkTikqG-VWkaSTGH3bb5JACcVZVcOR_A@mail.gmail.com>
	<4DF6A159.1010105@oddbird.net>
Message-ID: <E58B2502-0D1B-4DDE-A6B3-CD615BC18A3A@leidel.info>


On 14.06.2011, at 01:46, Carl Meyer wrote:

> On 06/13/2011 08:07 AM, Nick Coghlan wrote:
>> On Mon, Jun 13, 2011 at 10:50 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>>> You're right in terms of the current Python ecosystem and 3.x adoption, because
>>> of course this approach requires support from Python itself in terms of its
>>> site.py code. However, virtual environments have a utility beyond supporting
>>> older Pythons on newer OSes, since another common use case is having different
>>> library environments sandboxed from each other on different projects, even if
>>> all those projects are using Python 3.3+.
>> 
>> Yeah, even if the innate one struggles on later OS releases that
>> changed things in a backwards incompatible way, it will still be
>> valuable on the OS versions that are around at the time that version
>> of Python gets released.
> 
> FWIW, historically pretty much every new Python version has broken
> virtualenv, and new OS versions almost never have. Virtualenv isn't
> especially OS-dependent (not nearly as much as some other stdlib
> modules): the most OS-dependent piece is "shell activation", and that's
> a feature I would prefer to entirely leave out of the stdlib virtualenv
> (it's a convenience, not a necessity for virtualenv use, and the need to
> maintain it for a variety of OS shells is a maintenance burden I don't
> think Python should adopt).
> 
> In fact, the only new-OS-version adjustment I can recall virtualenv
> needing to make is when Debian introduced dist-packages -- but even that
> doesn't really apply, as that was distro-packager change to Python
> itself. With a built-in virtualenv it would be the distro packagers
> responsibility to make sure their patch to Python doesn't break the
> virtualenv module.

FTR, there is some special casing for Mac OS framework installs included, too.
Not sure if that should be considered a stability threatening issue though
since Apple hasn't changed much on that layout, AFAIK.

> So I don't think a virtualenv stdlib module would be at all likely to
> break on a new OS release, if Python itself is not broken by that OS
> release. (It certainly wouldn't be the stdlib module most likely to be
> broken by OS changes, in comparison to e.g. shutil, threading...)

Jannis

From ronaldoussoren at mac.com  Tue Jun 14 11:44:45 2011
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 14 Jun 2011 11:44:45 +0200
Subject: [Python-Dev] In-Python virtualisation and packaging
In-Reply-To: <E58B2502-0D1B-4DDE-A6B3-CD615BC18A3A@leidel.info>
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
	<BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>
	<loom.20110613T142559-463@post.gmane.org>
	<BANLkTikqG-VWkaSTGH3bb5JACcVZVcOR_A@mail.gmail.com>
	<4DF6A159.1010105@oddbird.net>
	<E58B2502-0D1B-4DDE-A6B3-CD615BC18A3A@leidel.info>
Message-ID: <B397B960-86DB-4775-A747-7BBE13E0E3BC@mac.com>


On 14 Jun, 2011, at 11:15, Jannis Leidel wrote:

> 
> On 14.06.2011, at 01:46, Carl Meyer wrote:
> 
>> 
>> In fact, the only new-OS-version adjustment I can recall virtualenv
>> needing to make is when Debian introduced dist-packages -- but even that
>> doesn't really apply, as that was distro-packager change to Python
>> itself. With a built-in virtualenv it would be the distro packagers
>> responsibility to make sure their patch to Python doesn't break the
>> virtualenv module.
> 
> FTR, there is some special casing for Mac OS framework installs included, too.
> Not sure if that should be considered a stability threatening issue though
> since Apple hasn't changed much on that layout, AFAIK.

Apple hasn't changed anything w.r.t. the basic layout of frameworks for a long time, but does mess with the structure of site-packages in their releases of Python. That shouldn't affect this feature though.

For the most part a Python.framework is just a unix install stuffed inside framework. The special-case code in virtualenv for frameworks is needed because a framework uses another mechanism to look for sys.prefix than a classical unix install: sys.prefix is the directory that contains the python shared library.

There is another feature of a framework install that would be nice to have in a virtualenv:  the python and pythonw commands for a framework build are small programs that use execv to start the real interpreter that's stored in a Python.app inside the framework. This is needed to be able to access GUI functionality from the command-line as Apple's GUI frameworks assume they are used by code in an application bundle. 

Ronald

From skip at montanaro.dyndns.org  Tue Jun 14 15:10:15 2011
From: skip at montanaro.dyndns.org (Skip Montanaro)
Date: Tue, 14 Jun 2011 08:10:15 -0500 (CDT)
Subject: [Python-Dev] Pathscale compilers open source
Message-ID: <20110614131015.91A3015F3A6A@montanaro.dyndns.org>

One of my colleagues with a background in the high performance computing
realm sent me this press release:

    http://www.pathscale.com/ekopath4-open-source-announcement

I'm not personally familiar with the Pathscale compilers, but thought some
folks here might be and might want to experiment with them.

Skip

From lukas.lueg at googlemail.com  Tue Jun 14 18:28:30 2011
From: lukas.lueg at googlemail.com (Lukas Lueg)
Date: Tue, 14 Jun 2011 18:28:30 +0200
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <4DF68709.8070206@canterbury.ac.nz>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
	<6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
	<4DF53B4D.1000607@canterbury.ac.nz>
	<E784DF2D-A8F9-4D81-998E-77F2905C9AB2@gmail.com>
	<4DF68709.8070206@canterbury.ac.nz>
Message-ID: <BANLkTikNArhQV+=V1K__Yv8qumETP6dPTA@mail.gmail.com>

> So I really can't see what harm it could do, except for
> maybe a tiny performance reduction in the case where you
> extract all the fields, or refer to extracted fields
> repeatedly.

Referring to the view-object multiple times should not be a problem
since the object can create and hold references to the unpacked values
it created; remember that they are all immutable.

From pje at telecommunity.com  Tue Jun 14 19:23:13 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 14 Jun 2011 13:23:13 -0400
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <20110614015606.1894.1336593159.divmod.xquotient.147@localh
	ost.localdomain>
References: <4DD2C2A5.3080403@stoneleaf.us>
	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>
	<4DD2D89D.4000303@stoneleaf.us>
	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>
	<4DD2F661.2050005@stoneleaf.us>
	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>
	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>
	<4DF4F4E0.5060003@stoneleaf.us> <4DF68B03.2010603@stoneleaf.us>
	<20110613234035.8D9A33A40D9@sparrow.telecommunity.com>
	<BANLkTinBS7kQjU-eEG7PoD=eMeRPx3PV+w@mail.gmail.com>
	<20110614015606.1894.1336593159.divmod.xquotient.147@localhost.localdomain>
Message-ID: <20110614172318.BE9203A4100@sparrow.telecommunity.com>

At 01:56 AM 6/14/2011 +0000, exarkun at twistedmatrix.com wrote:
>On 12:35 am, ncoghlan at gmail.com wrote:
>>On Tue, Jun 14, 2011 at 9:40 AM, P.J. Eby <pje at telecommunity.com> wrote:
>>>You can still do it one at a time:
>>>
>>>CHAR, = b'C'
>>>INT,  = b'I'
>>>...
>>>
>>>etc.  I just tried it with Python 3.1 and it works there.
>>
>>I almost mentioned that, although it does violate one of the
>>"unwritten rules of the Zen" (in this case, "syntax shall not look
>>like grit on Tim's monitor")
>
>    [CHAR] = b'C'
>    [INT]  = b'I'
>    ...

Holy carpal tunnel time machine...  That works in 2.3.  (Without the 
'b' of course.)  Didn't know you could just use list syntax like 
that.  It's an extra character to type, and two more shift keyings, 
but brevity isn't always the soul of clarity.


From ethan at stoneleaf.us  Tue Jun 14 19:46:42 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 14 Jun 2011 10:46:42 -0700
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <20110614172318.BE9203A4100@sparrow.telecommunity.com>
References: <4DD2C2A5.3080403@stoneleaf.us>	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>	<4DD2D89D.4000303@stoneleaf.us>	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>	<4DD2F661.2050005@stoneleaf.us>	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>	<4DF4F4E0.5060003@stoneleaf.us>
	<4DF68B03.2010603@stoneleaf.us>	<20110613234035.8D9A33A40D9@sparrow.telecommunity.com>	<BANLkTinBS7kQjU-eEG7PoD=eMeRPx3PV+w@mail.gmail.com>	<20110614015606.1894.1336593159.divmod.xquotient.147@localhost.localdomain>
	<20110614172318.BE9203A4100@sparrow.telecommunity.com>
Message-ID: <4DF79E82.7090208@stoneleaf.us>

P.J. Eby wrote:
> At 01:56 AM 6/14/2011 +0000, exarkun at twistedmatrix.com wrote:
>> On 12:35 am, ncoghlan at gmail.com wrote:
>>> On Tue, Jun 14, 2011 at 9:40 AM, P.J. Eby <pje at telecommunity.com> wrote:
>>>> You can still do it one at a time:
>>>>
>>>> CHAR, = b'C'
>>>> INT,  = b'I'
>>>> ...
>>>>
>>>> etc.  I just tried it with Python 3.1 and it works there.
>>>
>>> I almost mentioned that, although it does violate one of the
>>> "unwritten rules of the Zen" (in this case, "syntax shall not look
>>> like grit on Tim's monitor")
>>
>>    [CHAR] = b'C'
>>    [INT]  = b'I'
>>    ...
> 
> Holy carpal tunnel time machine...  That works in 2.3.  (Without the 'b' 
> of course.)  Didn't know you could just use list syntax like that.  It's 
> an extra character to type, and two more shift keyings, but brevity 
> isn't always the soul of clarity.

I'm thinking I like to the 'new' tuple-assignment character... ,= !

CHAR    ,= b'C'
DATE    ,= b'D'
LOGICAL ,= b'L'

;)

~Ethan~

From lukasz at langa.pl  Tue Jun 14 19:57:49 2011
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Tue, 14 Jun 2011 19:57:49 +0200
Subject: [Python-Dev] Python 3.x and bytes
In-Reply-To: <4DF79E82.7090208@stoneleaf.us>
References: <4DD2C2A5.3080403@stoneleaf.us>	<BANLkTimvzZeN5dmm2xHP5xV8Kpw2Nb9kuQ@mail.gmail.com>	<4DD2D89D.4000303@stoneleaf.us>	<BANLkTintqgBLFtBx8+1b+R10nywuKdKHOw@mail.gmail.com>	<4DD2F661.2050005@stoneleaf.us>	<BANLkTikx8U4jWwLXXazpmtsL6MasDmyVyg@mail.gmail.com>	<BANLkTik9oXD0Tont0MeyFF9im655946r2g@mail.gmail.com>	<4DF4F4E0.5060003@stoneleaf.us>
	<4DF68B03.2010603@stoneleaf.us>	<20110613234035.8D9A33A40D9@sparrow.telecommunity.com>	<BANLkTinBS7kQjU-eEG7PoD=eMeRPx3PV+w@mail.gmail.com>	<20110614015606.1894.1336593159.divmod.xquotient.147@localhost.localdomain>
	<20110614172318.BE9203A4100@sparrow.telecommunity.com>
	<4DF79E82.7090208@stoneleaf.us>
Message-ID: <6E9112BF-2100-46C7-871E-6EE0E4AE2049@langa.pl>

Wiadomo?? napisana przez Ethan Furman w dniu 2011-06-14, o godz. 19:46:

>>>   [CHAR] = b'C'
>>>   [INT]  = b'I'


> CHAR    ,= b'C'
> DATE    ,= b'D'
> LOGICAL ,= b'L'


Perl Jam!

-- 
Best regards,
?ukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110614/c2be5c6f/attachment.html>

From ndbecker2 at gmail.com  Tue Jun 14 20:26:58 2011
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 14 Jun 2011 14:26:58 -0400
Subject: [Python-Dev] Pathscale compilers open source
References: <20110614131015.91A3015F3A6A@montanaro.dyndns.org>
Message-ID: <it895i$k4q$1@dough.gmane.org>

Skip Montanaro wrote:

> One of my colleagues with a background in the high performance computing
> realm sent me this press release:
> 
>     http://www.pathscale.com/ekopath4-open-source-announcement
> 
> I'm not personally familiar with the Pathscale compilers, but thought some
> folks here might be and might want to experiment with them.
> 
> Skip

I just rebuilt all my c++ (boost::python) modules using pathscale, and I notice 
many crash with double-free on exit.  According to valgrind, this comes from the 
pathscale stl:

Just a heads-up.

==1927== Invalid free() / delete / delete[]
==1927==    at 0x4A0556E: free (vg_replace_malloc.c:366)
==1927==    by 0xDA77622: operator delete(void*) (in 
/home/nbecker/ekopath-4.0.10/lib/4.0.10/x8664/64/libcxxrt.so)
==1927==    by 0xD7BB91A: std::allocator<char>::deallocate(char*, unsigned long) 
(in /home/nbecker/ekopath-4.0.10/lib/4.0.10/x8664/64/libstl.so)
==1927==    by 0xD7BB99B: std::string::_C_unlink(char*) (in 
/home/nbecker/ekopath-4.0.10/lib/4.0.10/x8664/64/libstl.so)
==1927==    by 0xD7C4309: std::basic_string<char, std::char_traits<char>, 
std::allocator<char> >::~basic_string() (in 
/home/nbecker/ekopath-4.0.10/lib/4.0.10/x8664/64/libstl.so)
==1927==    by 0x3D64438940: __run_exit_handlers (in /lib64/libc-2.14.so)



From barry at python.org  Tue Jun 14 22:54:14 2011
From: barry at python.org (Barry Warsaw)
Date: Tue, 14 Jun 2011 16:54:14 -0400
Subject: [Python-Dev] packaging was not getting installed
Message-ID: <20110614165414.340f2138@neurotica.wooz.org>

I just fixed Makefile.pre.in to install the packaging directory and all its
subdirs.  Without this `python3.3 -c 'import packaging'` fails from the
installation target directory.  I'm not sure the Fellowship intends for every
subdir to get installed, so please double check.  I just added everything that
came from `find Lib/packaging -type d`.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110614/1ef6ad3b/attachment.pgp>

From nad at acm.org  Tue Jun 14 23:09:16 2011
From: nad at acm.org (Ned Deily)
Date: Tue, 14 Jun 2011 14:09:16 -0700
Subject: [Python-Dev] packaging was not getting installed
References: <20110614165414.340f2138@neurotica.wooz.org>
Message-ID: <nad-0C937F.14091614062011@news.gmane.org>

In article <20110614165414.340f2138 at neurotica.wooz.org>,
 Barry Warsaw <barry at python.org> wrote:
> I just fixed Makefile.pre.in to install the packaging directory and all its
> subdirs.  Without this `python3.3 -c 'import packaging'` fails from the
> installation target directory.  I'm not sure the Fellowship intends for every
> subdir to get installed, so please double check.  I just added everything that
> came from `find Lib/packaging -type d`.

http://bugs.python.org/issue12313

-- 
 Ned Deily,
 nad at acm.org


From merwok at netwok.org  Wed Jun 15 18:03:42 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 15 Jun 2011 18:03:42 +0200
Subject: [Python-Dev] Fwd: [Python-committers] Pulling from contributors
 repositories
In-Reply-To: <4DF5A1F0.2030207@m2.ccsnet.ne.jp>
References: <4DF5A1F0.2030207@m2.ccsnet.ne.jp>
Message-ID: <4DF8D7DE.4070002@netwok.org>

Le 13/06/2011 07:36, Hirokazu Yamamoto a ?crit :
> I've read the Python-committers thread "Pulling from contributors 
> repositories", which is about version control system. It seems there are 
> two main issues, linear (cleaner) history on pushing, and NEWS merging. 
> I'm newby of bazaar, but it seems to have a solution for first issue.

We are using Mercurial.

Regards

From merwok at netwok.org  Wed Jun 15 18:07:58 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 15 Jun 2011 18:07:58 +0200
Subject: [Python-Dev] pysetup as a top script
In-Reply-To: <BANLkTinHSg=YWQvDmY+QGxp5-K6rXJQK6A@mail.gmail.com>
References: <BANLkTimiPUBBPEzd04RpfDWAgTLL-HZnTQ@mail.gmail.com>	<BANLkTinzDKL9Fh5gq5mqu_fR5jVGqzYxsw@mail.gmail.com>
	<BANLkTinHSg=YWQvDmY+QGxp5-K6rXJQK6A@mail.gmail.com>
Message-ID: <4DF8D8DE.9050204@netwok.org>

Le 31/05/2011 08:45, Tarek Ziad? a ?crit :
> On Tue, May 31, 2011 at 7:13 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Tue, May 31, 2011 at 2:44 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>> If no one objects, I'll promote Tools/scripts/pysetup3 to a top level
>>> script that gets installed in scripts/ like 2to3, pydoc etc..
>>>
>>> That way, people will be able to use it directly when installing,
>>> removing projects, or studying what's installed
>> Cool.
>>
>> Now I'm trying to remember if it was a list discussion or the language
>> summit where you got the initial consensus on that approach...
> The thread starts here:
> http://mail.python.org/pipermail/python-dev/2010-October/104535.html
> 
> The pysetup top-level script was mentioned here:
> http://mail.python.org/pipermail/python-dev/2010-October/104581.html

A few other reasons that were not mentioned previously:

- In 2.4, we can?t run ?-m distutils2.run?, but a pysetup2.4 script works

- It?s nice for users to have something shorter than ?python3.3 -m
packaging.run run sdist? (I like to take ?make? as the ideal goal)

- It sends a message that we care about packaging (personal opinion)

Regards

From sandro.tosi at gmail.com  Wed Jun 15 18:42:35 2011
From: sandro.tosi at gmail.com (Sandro Tosi)
Date: Wed, 15 Jun 2011 18:42:35 +0200
Subject: [Python-Dev] Some additions to .hgignore
In-Reply-To: <4DE6193C.3000708@voidspace.org.uk>
References: <BANLkTinspoCeN5r4S3LW9=87NF6wEAkFRA@mail.gmail.com>
	<4DE6193C.3000708@voidspace.org.uk>
Message-ID: <BANLkTimoc43pzV1CmSEekwU9EJKTiUtc1Q@mail.gmail.com>

On 2011-06-01, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> That sounds good to me. An issue certainly wouldn't hurt.

So be it: http://bugs.python.org/issue12341 :)

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

From solipsis at pitrou.net  Thu Jun 16 00:21:40 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 16 Jun 2011 00:21:40 +0200
Subject: [Python-Dev] Parser/intrcheck.c
Message-ID: <20110616002140.77861fdd@pitrou.net>


Hello,

I may be missing something, but I'm wondering whether
Parser/intrcheck.c is still used anywhere.

It's only mentioned in some comments:

$ grep -r intrcheck.c *
Modules/signalmodule.c:1197:/* Replacements for intrcheck.c functionality
PC/os2vacpp/makefile.omk:217:  # intrcheck.c     -- Not Referenced by Anyone (?)
Python/sigcheck.c:3:   interrupt occurs.  It can't be in the intrcheck.c file since that

And if I remove it and "make clean", I can still rebuild successfully.

Regards

Antoine.



From mal at egenix.com  Thu Jun 16 19:18:45 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 16 Jun 2011 19:18:45 +0200
Subject: [Python-Dev] Python language summit on ustream.tv
Message-ID: <4DFA3AF5.3050306@egenix.com>

Dear Python Developers,

for the upcoming language summit at EuroPython, I'd like to
try out whether streaming such meetings would work. I'll setup
a webcam and stream the event live to a private channel on ustream.tv.

These are the details in case you want to watch:

URL: http://www.ustream.tv/channel/python-language-summit
PWD: fpmUtuL4

Date: Sunday, 2011-06-19
Time: 10:00 - 16:00 CEST with breaks

I'm not sure whether I can stream the whole summit, but at least
the morning session should be possible, provided the network
works on that day.

Interaction will likely be a bit difficult in case we have
heated discussions :-), but we'll keep the IRC channel
#python-language-summit on freenode open as well.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 16 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2011-05-23: Released eGenix mx Base 3.2.0      http://python.egenix.com/
2011-05-25: Released mxODBC 3.1.1              http://python.egenix.com/
2011-06-20: EuroPython 2011, Florence, Italy                4 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From db3l.net at gmail.com  Thu Jun 16 21:08:18 2011
From: db3l.net at gmail.com (David Bolen)
Date: Thu, 16 Jun 2011 15:08:18 -0400
Subject: [Python-Dev] Buildbot status web pages
Message-ID: <m2zklhvbkt.fsf@valheru.db3l.homeip.net>

I've been receiving 503 errors from the buildbot web status pages
beneath www.python.org/dev/buildbot for a day or two now - is there
perhaps something that needs a bit of a kick-start?

Thanks.

-- David


From ncoghlan at gmail.com  Fri Jun 17 07:20:44 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 17 Jun 2011 15:20:44 +1000
Subject: [Python-Dev] [Python-checkins] cpython (3.2): Fix typo.
In-Reply-To: <E1QXKBt-0001q6-87@dinsdale.python.org>
References: <E1QXKBt-0001q6-87@dinsdale.python.org>
Message-ID: <BANLkTi=0Kkg-9R=AW-F-y7Fk5UQ2NnAt6A@mail.gmail.com>

On Fri, Jun 17, 2011 at 7:33 AM, raymond.hettinger
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/0fe3b81c7c89
> changeset: ? 70822:0fe3b81c7c89
> branch: ? ? ?3.2
> parent: ? ? ?70819:8a9d25804c3d
> user: ? ? ? ?Raymond Hettinger <python at rcn.com>
> date: ? ? ? ?Thu Jun 16 22:32:10 2011 +0100
> summary:
> ?Fix typo.
>
> files:
> ?Doc/whatsnew/3.2.rst | ?2 +-
> ?1 files changed, 1 insertions(+), 1 deletions(-)
>
>
> diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst
> --- a/Doc/whatsnew/3.2.rst
> +++ b/Doc/whatsnew/3.2.rst
> @@ -385,7 +385,7 @@
> ?=====================================================
>
> ?This informational PEP clarifies how bytes/text issues are to be handled by the
> -WGSI protocol. ?The challenge is that string handling in Python 3 is most
> +WSGI protocol. ?The challenge is that string handling in Python 3 is most
> ?conveniently handled with the :class:`str` type even though the HTTP protocol
> ?is itself bytes oriented.

Ah, the cognitive scientist in me *loves* diffs like this one. Even
*knowing* there's a typo on the line, it's still hard to spot :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From lukas.lueg at googlemail.com  Fri Jun 17 16:44:12 2011
From: lukas.lueg at googlemail.com (Lukas Lueg)
Date: Fri, 17 Jun 2011 16:44:12 +0200
Subject: [Python-Dev] Lazy unpacking for struct module
In-Reply-To: <BANLkTikNArhQV+=V1K__Yv8qumETP6dPTA@mail.gmail.com>
References: <BANLkTimcd2jP0ZUziK0Eks+vWbhNGwRPaA@mail.gmail.com>
	<6656372E-7E82-477E-8C42-6CB686A1ADD1@gmail.com>
	<4DF53B4D.1000607@canterbury.ac.nz>
	<E784DF2D-A8F9-4D81-998E-77F2905C9AB2@gmail.com>
	<4DF68709.8070206@canterbury.ac.nz>
	<BANLkTikNArhQV+=V1K__Yv8qumETP6dPTA@mail.gmail.com>
Message-ID: <BANLkTi=mBBkEp5=+o8HDRz1M9g_4SKC71g@mail.gmail.com>

The followup: I've implemented a new StructView-object for 3.3a-trunk.
The object takes and existing Struct-object and provides on-access
unpacking. The breaking point where this object is faster than calling
Struct.unpack seems to be somewhere around 12 fields in the
format-string. Format strings with less fields expose too much
overhead of entering the C-code and staying there a little longer to
unpack all fields is actually faster.

Having fifteen or more fields in a format-string seems unlikely and
I'll therefor abandon the idea of providing this mechanism.

2011/6/14 Lukas Lueg <lukas.lueg at googlemail.com>:
>> So I really can't see what harm it could do, except for
>> maybe a tiny performance reduction in the case where you
>> extract all the fields, or refer to extracted fields
>> repeatedly.
>
> Referring to the view-object multiple times should not be a problem
> since the object can create and hold references to the unpacked values
> it created; remember that they are all immutable.
>

From status at bugs.python.org  Fri Jun 17 18:07:24 2011
From: status at bugs.python.org (Python tracker)
Date: Fri, 17 Jun 2011 18:07:24 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20110617160724.855FE1CCB6@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2011-06-10 - 2011-06-17)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2843 (+17)
  closed 21292 (+24)
  total  24135 (+41)

Open issues with patches: 1244 


Issues opened (29)
==================

#10884: pkgutil EggInfoDistribution requirements for .egg-info metadat
http://bugs.python.org/issue10884  reopened by eric.araujo

#12298: Sphinx glitch in library/functions
http://bugs.python.org/issue12298  reopened by eric.araujo

#12313: make install misses test dirs for packaging and email modules
http://bugs.python.org/issue12313  opened by eric.araujo

#12314: regrtest checks (os.environ, sys.path, etc.) are hard to use
http://bugs.python.org/issue12314  opened by eric.araujo

#12315: Improve http.client.HTTPResponse.read documentation
http://bugs.python.org/issue12315  opened by harobed

#12317: inspect.getabsfile() is not documented
http://bugs.python.org/issue12317  opened by giampaolo.rodola

#12319: [http.client] HTTPConnection.putrequest not support "chunked" 
http://bugs.python.org/issue12319  opened by harobed

#12321: documentation of ElementTree.find
http://bugs.python.org/issue12321  opened by patrick.vrijlandt

#12322: ElementPath 1.3 expressions documentation
http://bugs.python.org/issue12322  opened by patrick.vrijlandt

#12323: ElementPath 1.3 expressions
http://bugs.python.org/issue12323  opened by patrick.vrijlandt

#12326: Linux 3: tests should avoid using sys.platform == 'linux2'
http://bugs.python.org/issue12326  opened by neologix

#12327: in HTTPConnection the are len(body) and TypeError catch except
http://bugs.python.org/issue12327  opened by harobed

#12328: multiprocessing's overlapped PipeConnection on Windows
http://bugs.python.org/issue12328  opened by sbt

#12331: lib2to3 and packaging tests fail because they write into prote
http://bugs.python.org/issue12331  opened by vinay.sajip

#12335: pysetup create: ask before moving an existing setup.cfg
http://bugs.python.org/issue12335  opened by barry

#12336: tkinter.test.test_ttk.test_widgets.test_select() failure on Fr
http://bugs.python.org/issue12336  opened by haypo

#12337: Need real TextIOWrapper for stdin/stdout
http://bugs.python.org/issue12337  opened by r_mosaic

#12338: multiprocessing.util._eintr_retry doen't recalculate timeouts
http://bugs.python.org/issue12338  opened by sbt

#12340: Access violation when using the C version of the io module
http://bugs.python.org/issue12340  opened by OscarL

#12341: Some additions to .hgignore
http://bugs.python.org/issue12341  opened by sandro.tosi

#12342: characters with ord above 65535 fail to display in IDLE
http://bugs.python.org/issue12342  opened by wujek.srujek

#12343: ssl documentation needs comments about non-blocking behaviour
http://bugs.python.org/issue12343  opened by jcea

#12344: Add **kwargs to get_reinitialized_command
http://bugs.python.org/issue12344  opened by higery

#12346: Python 2.7.2 source code build (release) depends on mercurial
http://bugs.python.org/issue12346  opened by Graeme.Winter

#12347: add an "extras" in packaging.pypi.simple.Crawler
http://bugs.python.org/issue12347  opened by tarek

#12348: case sensitivness  in packaging.pypi.simple.Crawler
http://bugs.python.org/issue12348  opened by tarek

#12350: Improve stat_result.st_blocks and st_blksize documentation
http://bugs.python.org/issue12350  opened by liw

#12352: multiprocessing.Value() hangs
http://bugs.python.org/issue12352  opened by greg.ath

#12353: argparse cannot handle empty arguments
http://bugs.python.org/issue12353  opened by bjacobs



Most recent 15 issues with no replies (15)
==========================================

#12353: argparse cannot handle empty arguments
http://bugs.python.org/issue12353

#12352: multiprocessing.Value() hangs
http://bugs.python.org/issue12352

#12347: add an "extras" in packaging.pypi.simple.Crawler
http://bugs.python.org/issue12347

#12338: multiprocessing.util._eintr_retry doen't recalculate timeouts
http://bugs.python.org/issue12338

#12336: tkinter.test.test_ttk.test_widgets.test_select() failure on Fr
http://bugs.python.org/issue12336

#12323: ElementPath 1.3 expressions
http://bugs.python.org/issue12323

#12322: ElementPath 1.3 expressions documentation
http://bugs.python.org/issue12322

#12321: documentation of ElementTree.find
http://bugs.python.org/issue12321

#12317: inspect.getabsfile() is not documented
http://bugs.python.org/issue12317

#12315: Improve http.client.HTTPResponse.read documentation
http://bugs.python.org/issue12315

#12303: expose sigwaitinfo() and sigtimedwait() in the signal module
http://bugs.python.org/issue12303

#12297: Clarifications to atexit.register and unregister doc
http://bugs.python.org/issue12297

#12296: Minor clarification in devguide
http://bugs.python.org/issue12296

#12295: Fix ResourceWarning in turtledemo help window
http://bugs.python.org/issue12295

#12294: multiprocessing.Pool: Need a way to find out if work are finis
http://bugs.python.org/issue12294



Most recent 15 issues waiting for review (15)
=============================================

#12350: Improve stat_result.st_blocks and st_blksize documentation
http://bugs.python.org/issue12350

#12344: Add **kwargs to get_reinitialized_command
http://bugs.python.org/issue12344

#12341: Some additions to .hgignore
http://bugs.python.org/issue12341

#12328: multiprocessing's overlapped PipeConnection on Windows
http://bugs.python.org/issue12328

#12315: Improve http.client.HTTPResponse.read documentation
http://bugs.python.org/issue12315

#12313: make install misses test dirs for packaging and email modules
http://bugs.python.org/issue12313

#12308: Add link to PEP 0 for topical index in wiki
http://bugs.python.org/issue12308

#12306: zlib: Expose zlibVersion to query runtime version of zlib
http://bugs.python.org/issue12306

#12304: expose signalfd(2) in the signal module
http://bugs.python.org/issue12304

#12297: Clarifications to atexit.register and unregister doc
http://bugs.python.org/issue12297

#12296: Minor clarification in devguide
http://bugs.python.org/issue12296

#12295: Fix ResourceWarning in turtledemo help window
http://bugs.python.org/issue12295

#12291: file written using marshal in 3.2 can be read by 2.7, but not 
http://bugs.python.org/issue12291

#12290: __setstate__ is called for false values
http://bugs.python.org/issue12290

#12289: http.server.CGIHTTPRequestHandler doesn't check if a Python sc
http://bugs.python.org/issue12289



Top 10 most discussed issues (10)
=================================

#11610: Improved support for abstract base classes with descriptors
http://bugs.python.org/issue11610  15 msgs

#12313: make install misses test dirs for packaging and email modules
http://bugs.python.org/issue12313  13 msgs

#12326: Linux 3: tests should avoid using sys.platform == 'linux2'
http://bugs.python.org/issue12326  12 msgs

#12343: ssl documentation needs comments about non-blocking behaviour
http://bugs.python.org/issue12343  12 msgs

#12281: bytes.decode('mbcs', 'ignore') does replace undecodable bytes 
http://bugs.python.org/issue12281   9 msgs

#11553: Docs for: import, packages, site.py, .pth files
http://bugs.python.org/issue11553   7 msgs

#12167: test_packaging reference leak
http://bugs.python.org/issue12167   7 msgs

#12287: ossaudiodev: stack corruption with FD >= FD_SETSIZE
http://bugs.python.org/issue12287   7 msgs

#12328: multiprocessing's overlapped PipeConnection on Windows
http://bugs.python.org/issue12328   7 msgs

#9302: distutils API Reference: setup() and Extension parameters' des
http://bugs.python.org/issue9302   6 msgs



Issues closed (26)
==================

#9284: inspect.findsource() cannot find source for doctest code
http://bugs.python.org/issue9284  closed by python-dev

#10457: "Related help topics" shown outside pager
http://bugs.python.org/issue10457  closed by eric.araujo

#11595: Miscellaneous bugs in cfg_to_args() utility function
http://bugs.python.org/issue11595  closed by eric.araujo

#12009: netrc module crashes if netrc file has comment lines
http://bugs.python.org/issue12009  closed by python-dev

#12084: os.stat() on windows doesn't consider relative symlink
http://bugs.python.org/issue12084  closed by brian.curtin

#12133: ResourceWarning in urllib.request
http://bugs.python.org/issue12133  closed by haypo

#12206: Documentation Std. Library 15.7.5 "LogRecord objects": Paramet
http://bugs.python.org/issue12206  closed by python-dev

#12240: Allow multiple setup_hooks
http://bugs.python.org/issue12240  closed by eric.araujo

#12246: Warn when trying to install third-party module from an uninsta
http://bugs.python.org/issue12246  closed by eric.araujo

#12248: __dir__ semantics changed in Python 2.7.2
http://bugs.python.org/issue12248  closed by benjamin.peterson

#12310: Segfault in test_multiprocessing
http://bugs.python.org/issue12310  closed by haypo

#12316: test_signal: test_sigwait_thread failure on FreeBSD 6.4 buildb
http://bugs.python.org/issue12316  closed by haypo

#12318: list + tuple inconsistency
http://bugs.python.org/issue12318  closed by rhettinger

#12320: test_packaging failures
http://bugs.python.org/issue12320  closed by haypo

#12324: [3.2] sorted(big dict)
http://bugs.python.org/issue12324  closed by mark.dickinson

#12325: regex matches incorrectly on literal dot (99.9% confirmed)
http://bugs.python.org/issue12325  closed by r.david.murray

#12329: XHTML entity apos missing in htmlentitydefs
http://bugs.python.org/issue12329  closed by eric.araujo

#12330: Named set methods on collections.Set and dict view objects
http://bugs.python.org/issue12330  closed by Julian

#12332: Float division
http://bugs.python.org/issue12332  closed by benjamin.peterson

#12333: test_packaging failures under Solaris
http://bugs.python.org/issue12333  closed by haypo

#12334: Strange sort error
http://bugs.python.org/issue12334  closed by rengel

#12339: logging.Formatter.format() assumes exception to support str() 
http://bugs.python.org/issue12339  closed by vinay.sajip

#12345: Add math.tau
http://bugs.python.org/issue12345  closed by pitrou

#12349: Typo in 3.2 "What's New": WGSI / WSGI
http://bugs.python.org/issue12349  closed by sandro.tosi

#12351: Update URL for pycrypto project
http://bugs.python.org/issue12351  closed by python-dev

#1711800: SequenceMatcher bug with insert/delete block after "replace"
http://bugs.python.org/issue1711800  closed by terry.reedy

From martin at v.loewis.de  Sun Jun 19 13:07:14 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 19 Jun 2011 13:07:14 +0200
Subject: [Python-Dev] [Python-checkins] cpython: #11781: update windows
 build script to account for move of email tests
In-Reply-To: <4DFD07EF.6060405@udel.edu>
References: <E1QXxiz-00050C-Tp@dinsdale.python.org> <4DFD07EF.6060405@udel.edu>
Message-ID: <4DFDD862.9030709@v.loewis.de>

>>               # This should contain all non-.svn files listed in
>> subversion
> 
> Should this be mercurial

Perhaps.

> 
>>               for f in os.listdir(lib.absolute):
>>                   if f.endswith(".txt") or f==".svn":continue
> 
> and this .hg?

No.

Martin

From victor.stinner at haypocalc.com  Sun Jun 19 16:17:04 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 19 Jun 2011 16:17:04 +0200
Subject: [Python-Dev] [Python-checkins] cpython: edit and rewrite
In-Reply-To: <E1QXjl5-000882-29@dinsdale.python.org>
References: <E1QXjl5-000882-29@dinsdale.python.org>
Message-ID: <1308493024.30115.6.camel@marge>

Le samedi 18 juin 2011 ? 02:51 +0200, benjamin.peterson a ?crit :
>  .. function:: dump_traceback(file=sys.stderr, all_threads=True)
>  
> -   Dump the traceback of all threads, or of the current thread if *all_threads*
> -   is ``False``, into *file*.
> +   Dump the traceback of all threads into *file*. If *all_threads* is ``True``,
> +   produce tracebacks for every running thread. Otherwise, dump only the current
> +   thread.

The first sentence is confusing. I suggest:

Dump the traceback of all threads into *file*. If *all_threads* is
``False``, dump the traceback of only the current thread.

or

Dump the traceback into *file*. If *all_threads* is ``True``, produce
tracebacks for every running thread. Otherwise, dump only the current
thread.

> @@ -69,15 +75,14 @@
>  .. function:: dump_tracebacks_later(timeout, repeat=False, file=sys.stderr, exit=False)
>  
>     Dump the tracebacks of all threads, after a timeout of *timeout* seconds, or
> -   each *timeout* seconds if *repeat* is ``True``.  If *exit* is True, call
> -   :c:func:`_exit` with status=1 after dumping the tracebacks to terminate
> -   immediatly the process, which is not safe.  For example, :c:func:`_exit`
> -   doesn't flush file buffers.  If the function is called twice, the new call
> -   replaces previous parameters (resets the timeout). The timer has a
> -   sub-second resolution.
> +   every *timeout* seconds if *repeat* is ``True``.  If *exit* is ``True``, call
> +   :c:func:`_exit` with status=1 after dumping the tracebacks.  (Note
> +   :c:func:`_exit` doesn't flush file buffers.) If the function is called twice,
> +   the new call replaces previous parameters and resets the timeout. The timer
> +   has a sub-second resolution.

You removed "to terminate immediatly the process, which is not safe"
sentence which is very important. It doesn't exit like sys.exit(): it
exits immediatly. Not flushing file buffers is just an example.

Anyway, thank you for rephrasing the doc.

Victor


From ncoghlan at gmail.com  Sun Jun 19 17:18:03 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 20 Jun 2011 01:18:03 +1000
Subject: [Python-Dev] [Python-checkins] devguide: Fix copy-paste glitch
	:)
In-Reply-To: <E1QYJQt-0002S4-F6@dinsdale.python.org>
References: <E1QYJQt-0002S4-F6@dinsdale.python.org>
Message-ID: <BANLkTinaaEAv-Shzi+eCG_0qJgoNAFQ6bg@mail.gmail.com>

On Mon, Jun 20, 2011 at 12:57 AM, eric.araujo
<python-checkins at python.org> wrote:
> http://hg.python.org/devguide/rev/f55ad3dc4526
> changeset: ? 434:f55ad3dc4526
> user: ? ? ? ??ric Araujo <merwok at netwok.org>
> date: ? ? ? ?Sun Jun 19 16:56:58 2011 +0200
> summary:
> ?Fix copy-paste glitch :)
>
> files:
> ?committing.rst | ?2 +-
> ?1 files changed, 1 insertions(+), 1 deletions(-)
>
>
> diff --git a/committing.rst b/committing.rst
> --- a/committing.rst
> +++ b/committing.rst
> @@ -16,7 +16,7 @@
> ? (using Tools/scripts/reindent-rst.py)
> ?* Has the documentation been updated?
> ?* Has the test suite been updated?
> -* Has ``Misc/ACKS`` been updated?
> +* Has ``Misc/NEWS`` been updated?
> ?* Has ``Misc/ACKS`` been updated?
> ?* Has the test suite been run?

D'oh! I had it right at one point, too, but then I rearranged it and
must have clobbered the wrong line in the process. Thanks for fixing
it :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From victor.stinner at haypocalc.com  Sun Jun 19 19:18:15 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 19 Jun 2011 19:18:15 +0200
Subject: [Python-Dev] [Python-checkins] devguide: Add a communications
 section to the devguide FAQ (closes #11690)
In-Reply-To: <E1QYJLt-00027Q-Ia@dinsdale.python.org>
References: <E1QYJLt-00027Q-Ia@dinsdale.python.org>
Message-ID: <1308503895.30320.2.camel@marge>

Le dimanche 19 juin 2011 ? 16:51 +0200, nick.coghlan a ?crit :
> +Where should I ask general Python questions?
> +--------------------------------------------
> +
> +General Python questions should still go to `python-list`_ or `python-tutor`_
> +or similar resources, such as StackOverflow_ or ``#python`` on IRC.

You should mention the IRC server, I suppose that you are talking about
the Freenode server.

There are other channels in other languages, like #python-fr (on
Freenode).

You may also add a reference to the #python-dev chanel. I am connected
most of the time, but not always available.

Victor


From jimjjewett at gmail.com  Sun Jun 19 21:40:01 2011
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 19 Jun 2011 15:40:01 -0400
Subject: [Python-Dev] [Python-checkins] cpython: #6771: Move wrapper
 function into __init__ and eliminate wrapper module
In-Reply-To: <E1QY5lX-0001bo-VC@dinsdale.python.org>
References: <E1QY5lX-0001bo-VC@dinsdale.python.org>
Message-ID: <BANLkTi=4xsa2S-xD5eu7zQxnQzESxHLJ2A@mail.gmail.com>

Does this really need to be a bare except?

On Sat, Jun 18, 2011 at 8:21 PM, r.david.murray
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/9c96c3adbcd1
> changeset: ? 70867:9c96c3adbcd1
> user: ? ? ? ?R David Murray <rdmurray at bitdance.com>
> date: ? ? ? ?Sat Jun 18 20:21:09 2011 -0400
> summary:
> ?#6771: Move wrapper function into __init__ and eliminate wrapper module
>
> Andrew agreed in the issue that eliminating the module file made sense.
> Wrapper has only been exposed as a function, and so there is no (easy)
> way to access the wrapper module, which in any case only had the one
> function in it. ?Since __init__ already contains a couple wrapper
> functions, it seems to make sense to just move wrapper there instead of
> importing it from a single function module.
>
> files:
> ?Lib/curses/__init__.py | ?46 +++++++++++++++++++++++++++-
> ?Lib/curses/wrapper.py ?| ?50 ------------------------------
> ?Misc/NEWS ? ? ? ? ? ? ?| ? 4 ++
> ?3 files changed, 49 insertions(+), 51 deletions(-)
>
>
> diff --git a/Lib/curses/__init__.py b/Lib/curses/__init__.py
> --- a/Lib/curses/__init__.py
> +++ b/Lib/curses/__init__.py
> @@ -13,7 +13,6 @@
> ?__revision__ = "$Id$"
>
> ?from _curses import *
> -from curses.wrapper import wrapper
> ?import os as _os
> ?import sys as _sys
>
> @@ -57,3 +56,48 @@
> ? ? has_key
> ?except NameError:
> ? ? from has_key import has_key
> +
> +# Wrapper for the entire curses-based application. ?Runs a function which
> +# should be the rest of your curses-based application. ?If the application
> +# raises an exception, wrapper() will restore the terminal to a sane state so
> +# you can read the resulting traceback.
> +
> +def wrapper(func, *args, **kwds):
> + ? ?"""Wrapper function that initializes curses and calls another function,
> + ? ?restoring normal keyboard/screen behavior on error.
> + ? ?The callable object 'func' is then passed the main window 'stdscr'
> + ? ?as its first argument, followed by any other arguments passed to
> + ? ?wrapper().
> + ? ?"""
> +
> + ? ?try:
> + ? ? ? ?# Initialize curses
> + ? ? ? ?stdscr = initscr()
> +
> + ? ? ? ?# Turn off echoing of keys, and enter cbreak mode,
> + ? ? ? ?# where no buffering is performed on keyboard input
> + ? ? ? ?noecho()
> + ? ? ? ?cbreak()
> +
> + ? ? ? ?# In keypad mode, escape sequences for special keys
> + ? ? ? ?# (like the cursor keys) will be interpreted and
> + ? ? ? ?# a special value like curses.KEY_LEFT will be returned
> + ? ? ? ?stdscr.keypad(1)
> +
> + ? ? ? ?# Start color, too. ?Harmless if the terminal doesn't have
> + ? ? ? ?# color; user can test with has_color() later on. ?The try/catch
> + ? ? ? ?# works around a minor bit of over-conscientiousness in the curses
> + ? ? ? ?# module -- the error return from C start_color() is ignorable.
> + ? ? ? ?try:
> + ? ? ? ? ? ?start_color()
> + ? ? ? ?except:
> + ? ? ? ? ? ?pass
> +
> + ? ? ? ?return func(stdscr, *args, **kwds)
> + ? ?finally:
> + ? ? ? ?# Set everything back to normal
> + ? ? ? ?if 'stdscr' in locals():
> + ? ? ? ? ? ?stdscr.keypad(0)
> + ? ? ? ? ? ?echo()
> + ? ? ? ? ? ?nocbreak()
> + ? ? ? ? ? ?endwin()
> diff --git a/Lib/curses/wrapper.py b/Lib/curses/wrapper.py
> deleted file mode 100644
> --- a/Lib/curses/wrapper.py
> +++ /dev/null
> @@ -1,50 +0,0 @@
> -"""curses.wrapper
> -
> -Contains one function, wrapper(), which runs another function which
> -should be the rest of your curses-based application. ?If the
> -application raises an exception, wrapper() will restore the terminal
> -to a sane state so you can read the resulting traceback.
> -
> -"""
> -
> -import curses
> -
> -def wrapper(func, *args, **kwds):
> - ? ?"""Wrapper function that initializes curses and calls another function,
> - ? ?restoring normal keyboard/screen behavior on error.
> - ? ?The callable object 'func' is then passed the main window 'stdscr'
> - ? ?as its first argument, followed by any other arguments passed to
> - ? ?wrapper().
> - ? ?"""
> -
> - ? ?try:
> - ? ? ? ?# Initialize curses
> - ? ? ? ?stdscr = curses.initscr()
> -
> - ? ? ? ?# Turn off echoing of keys, and enter cbreak mode,
> - ? ? ? ?# where no buffering is performed on keyboard input
> - ? ? ? ?curses.noecho()
> - ? ? ? ?curses.cbreak()
> -
> - ? ? ? ?# In keypad mode, escape sequences for special keys
> - ? ? ? ?# (like the cursor keys) will be interpreted and
> - ? ? ? ?# a special value like curses.KEY_LEFT will be returned
> - ? ? ? ?stdscr.keypad(1)
> -
> - ? ? ? ?# Start color, too. ?Harmless if the terminal doesn't have
> - ? ? ? ?# color; user can test with has_color() later on. ?The try/catch
> - ? ? ? ?# works around a minor bit of over-conscientiousness in the curses
> - ? ? ? ?# module -- the error return from C start_color() is ignorable.
> - ? ? ? ?try:
> - ? ? ? ? ? ?curses.start_color()
> - ? ? ? ?except:
> - ? ? ? ? ? ?pass
> -
> - ? ? ? ?return func(stdscr, *args, **kwds)
> - ? ?finally:
> - ? ? ? ?# Set everything back to normal
> - ? ? ? ?if 'stdscr' in locals():
> - ? ? ? ? ? ?stdscr.keypad(0)
> - ? ? ? ? ? ?curses.echo()
> - ? ? ? ? ? ?curses.nocbreak()
> - ? ? ? ? ? ?curses.endwin()
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -193,6 +193,10 @@
> ?Library
> ?-------
>
> +- Issue #6771: moved the curses.wrapper function from the single-function
> + ?wrapper module into __init__, eliminating the module. ?Since __init__ was
> + ?already importing the function to curses.wrapper, there is no API change.
> +
> ?- Issue #11584: email.header.decode_header no longer fails if the header
> ? passed to it is a Header object, and Header/make_header no longer fail
> ? if given binary unknown-8bit input.
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>

From merwok at netwok.org  Sun Jun 19 22:59:26 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Sun, 19 Jun 2011 22:59:26 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.1): Fix closes
 issue12261 - Minor documention changes in the urllib.parse.rst
In-Reply-To: <E1QYP3P-0001XD-73@dinsdale.python.org>
References: <E1QYP3P-0001XD-73@dinsdale.python.org>
Message-ID: <4DFE632E.2010308@netwok.org>

Hi,

Remember that 3.1 is in security mode, and as such will not get new
documentation releases.  See the previous threads about 2.6 docs or
security releases for more info.

Regards

From senthil at uthcode.com  Sun Jun 19 23:41:22 2011
From: senthil at uthcode.com (Senthil Kumaran)
Date: Sun, 19 Jun 2011 14:41:22 -0700
Subject: [Python-Dev] [Python-checkins] cpython (3.1): Fix closes
 issue12261 - Minor documention changes in the urllib.parse.rst
In-Reply-To: <4DFE632E.2010308@netwok.org>
References: <E1QYP3P-0001XD-73@dinsdale.python.org>
	<4DFE632E.2010308@netwok.org>
Message-ID: <20110619214122.GA3964@mathmagic>

On Sun, Jun 19, 2011 at 10:59:26PM +0200, ?ric Araujo wrote:
> 
> Remember that 3.1 is in security mode, and as such will not get new
> documentation releases.  See the previous threads about 2.6 docs or
> security releases for more info.

Thanks for the information. I missed that somehow. Noted and that
check that thread.

-- 
Senthil

From rdmurray at bitdance.com  Mon Jun 20 04:39:32 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 19 Jun 2011 22:39:32 -0400
Subject: [Python-Dev] [Python-checkins] cpython: #6771: Move wrapper
	function into __init__ and eliminate wrapper module
In-Reply-To: <BANLkTi=4xsa2S-xD5eu7zQxnQzESxHLJ2A@mail.gmail.com>
References: <E1QY5lX-0001bo-VC@dinsdale.python.org>
	<BANLkTi=4xsa2S-xD5eu7zQxnQzESxHLJ2A@mail.gmail.com>
Message-ID: <20110620023933.0ABC1250D3A@webabinitio.net>

On Sun, 19 Jun 2011 15:40:01 -0400, Jim Jewett <jimjjewett at gmail.com> wrote:
> Does this really need to be a bare except?

No, but that's a separate bug report, which you are welcome to
file.  The issue I closed was about moving the existing code.

--
R. David Murray           http://www.bitdance.com

From g.brandl at gmx.net  Mon Jun 20 09:11:20 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 20 Jun 2011 09:11:20 +0200
Subject: [Python-Dev] cpython (3.2): Fix closes Issue12315 - Updates to
 http.client documentation.
In-Reply-To: <E1QYRv2-0004I2-2S@dinsdale.python.org>
References: <E1QYRv2-0004I2-2S@dinsdale.python.org>
Message-ID: <itmrpp$rqu$1@dough.gmane.org>

On 20.06.2011 02:00, senthil.kumaran wrote:
> http://hg.python.org/cpython/rev/d801b570b1dd
> changeset:   70882:d801b570b1dd
> branch:      3.2
> parent:      70880:9e58a638f028
> user:        Senthil Kumaran <senthil at uthcode.com>
> date:        Sun Jun 19 16:56:49 2011 -0700
> summary:
>   Fix closes Issue12315 - Updates to http.client documentation.
> 
> files:
>   Doc/library/http.client.rst |  13 ++++++++++++-
>   1 files changed, 12 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
> --- a/Doc/library/http.client.rst
> +++ b/Doc/library/http.client.rst
> @@ -543,6 +543,9 @@
>     A debugging hook.  If :attr:`debuglevel` is greater than zero, messages
>     will be printed to stdout as the response is read and parsed.
>  
> +.. attribute:: HTTPResponse.closed
> +
> +   Is True if the stream is closed. 
>
>  Examples
>  --------

This is not a big deal, and I'm not picking specially on you here, Senthil,
it's just something that I've noticed several times:

Newlines are a valuable tool for structuring reST files (just like in Python
files).  I tried to set up a convention to separate large blocks (such as
sections) by two newlines, to make it easier to scroll and find what you're
looking for.  Please try to keep this intact.

Thanks,
Georg


From g.brandl at gmx.net  Mon Jun 20 09:13:07 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 20 Jun 2011 09:13:07 +0200
Subject: [Python-Dev] cpython (3.2): Fix closes Issue12359 - Minor
 update to module import description.
In-Reply-To: <E1QYSZW-0006br-I7@dinsdale.python.org>
References: <E1QYSZW-0006br-I7@dinsdale.python.org>
Message-ID: <itmrt4$rqu$2@dough.gmane.org>

On 20.06.2011 02:42, senthil.kumaran wrote:
> http://hg.python.org/cpython/rev/bf8b4c43fb94
> changeset:   70886:bf8b4c43fb94
> branch:      3.2
> parent:      70884:4444afcfb22e
> user:        Senthil Kumaran <senthil at uthcode.com>
> date:        Sun Jun 19 17:37:06 2011 -0700
> summary:
>   Fix closes Issue12359 - Minor update to module import description.
> 
> files:
>   Doc/tutorial/modules.rst |  15 ++++++++-------
>   1 files changed, 8 insertions(+), 7 deletions(-)
> 
> 
> diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst
> --- a/Doc/tutorial/modules.rst
> +++ b/Doc/tutorial/modules.rst
> @@ -159,13 +159,14 @@
>  
>  .. index:: triple: module; search; path
>  
> -When a module named :mod:`spam` is imported, the interpreter searches for a file
> -named :file:`spam.py` in the current directory, and then in the list of
> -directories specified by the environment variable :envvar:`PYTHONPATH`.  This
> -has the same syntax as the shell variable :envvar:`PATH`, that is, a list of
> -directory names.  When :envvar:`PYTHONPATH` is not set, or when the file is not
> -found there, the search continues in an installation-dependent default path; on
> -Unix, this is usually :file:`.:/usr/local/lib/python`.
> +When a module named :mod:`spam` is imported, the interpreter searches for a
> +file named :file:`spam.py` in the directory containing the input script (or
> +thecurrent directory), and then in the list of directories specified by the

By just adding "or the current directory", you've actually made this more
confusing: now the reader will wonder when it's the script directory and when
it's the current directory.

Georg


From senthil at uthcode.com  Mon Jun 20 09:34:07 2011
From: senthil at uthcode.com (Senthil Kumaran)
Date: Mon, 20 Jun 2011 00:34:07 -0700
Subject: [Python-Dev] cpython (3.2): Fix closes Issue12359 - Minor
 update to module import description.
In-Reply-To: <itmrt4$rqu$2@dough.gmane.org>
References: <E1QYSZW-0006br-I7@dinsdale.python.org>
	<itmrt4$rqu$2@dough.gmane.org>
Message-ID: <20110620073407.GA9220@mathmagic>

On Mon, Jun 20, 2011 at 09:13:07AM +0200, Georg Brandl wrote:
> On 20.06.2011 02:42, senthil.kumaran wrote:
> > summary:
> >   Fix closes Issue12359 - Minor update to module import description.
> > ...
> > +When a module named :mod:`spam` is imported, the interpreter searches for a
> > +file named :file:`spam.py` in the directory containing the input script (or
> > +thecurrent directory), and then in the list of directories specified by the
> 
> By just adding "or the current directory", you've actually made this more
> confusing: now the reader will wonder when it's the script directory and when
> it's the current directory.

I added that statement in the bracket, after looking at another
instance in the next para which had this mention. I think, the point
here is that the reader would understand, where the import is looking
for the module based on the context.  Fine with removing this sentence
("or the current directory"), if statement explains the idea better
without it.

-- 
Senthil

From senthil at uthcode.com  Mon Jun 20 09:36:02 2011
From: senthil at uthcode.com (Senthil Kumaran)
Date: Mon, 20 Jun 2011 00:36:02 -0700
Subject: [Python-Dev] cpython (3.2): Fix closes Issue12315 - Updates to
 http.client documentation.
In-Reply-To: <itmrpp$rqu$1@dough.gmane.org>
References: <E1QYRv2-0004I2-2S@dinsdale.python.org>
	<itmrpp$rqu$1@dough.gmane.org>
Message-ID: <20110620073602.GB9220@mathmagic>

On Mon, Jun 20, 2011 at 09:11:20AM +0200, Georg Brandl wrote:
> Newlines are a valuable tool for structuring reST files (just like in Python
> files).  I tried to set up a convention to separate large blocks (such as
> sections) by two newlines, to make it easier to scroll and find what you're
> looking for.  Please try to keep this intact.
> 

Noted. In the next checkin to this file, I shall correct this one and
add extra line before the Example section. 

Thanks,
Senthil

From fijall at gmail.com  Mon Jun 20 10:08:04 2011
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 20 Jun 2011 10:08:04 +0200
Subject: [Python-Dev] Language summit writeup anyone?
Message-ID: <BANLkTinwD=9hK0N=q0vxf9d-i9ht1xOiOQ@mail.gmail.com>

Hi.

Unfortunately I'm missing Europython (and language summit) this year.
Did anyone do a writeup on what was discussed?

Cheers,
fijal

From doug.hellmann at gmail.com  Mon Jun 20 13:20:44 2011
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Mon, 20 Jun 2011 07:20:44 -0400
Subject: [Python-Dev] Language summit writeup anyone?
In-Reply-To: <BANLkTinwD=9hK0N=q0vxf9d-i9ht1xOiOQ@mail.gmail.com>
References: <BANLkTinwD=9hK0N=q0vxf9d-i9ht1xOiOQ@mail.gmail.com>
Message-ID: <F2D3CD03-8C37-4DBC-8784-9092E9997A03@gmail.com>


On Jun 20, 2011, at 4:08 AM, Maciej Fijalkowski wrote:

> Hi.
> 
> Unfortunately I'm missing Europython (and language summit) this year.
> Did anyone do a writeup on what was discussed?

Brian Curtin or I can help get the writeup posted to the Python Insider blog. I'm sure there are a lot of people who don't follow this list who would be interested in hearing about the outcome.

Doug


From solipsis at pitrou.net  Mon Jun 20 13:31:31 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 20 Jun 2011 13:31:31 +0200
Subject: [Python-Dev] Language summit writeup anyone?
In-Reply-To: <BANLkTinwD=9hK0N=q0vxf9d-i9ht1xOiOQ@mail.gmail.com>
References: <BANLkTinwD=9hK0N=q0vxf9d-i9ht1xOiOQ@mail.gmail.com>
Message-ID: <20110620133131.553e4c87@msiwind>

Hi,

Le Mon, 20 Jun 2011 10:08:04 +0200,
Maciej Fijalkowski <fijall at gmail.com> a ?crit :
> 
> Unfortunately I'm missing Europython (and language summit) this year.
> Did anyone do a writeup on what was discussed?

Mark Dickinson has been taking notes, but since there only a few of us
(roughly 10 attendants), it was mostly casual and friendly chatting :)

Regards

Antoine.



From g.brandl at gmx.net  Mon Jun 20 17:37:55 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 20 Jun 2011 17:37:55 +0200
Subject: [Python-Dev] cpython (3.2): Fix closes Issue12359 - Minor
 update to module import description.
In-Reply-To: <20110620073407.GA9220@mathmagic>
References: <E1QYSZW-0006br-I7@dinsdale.python.org>
	<itmrt4$rqu$2@dough.gmane.org> <20110620073407.GA9220@mathmagic>
Message-ID: <itnpfj$ltd$1@dough.gmane.org>

On 20.06.2011 09:34, Senthil Kumaran wrote:
> On Mon, Jun 20, 2011 at 09:13:07AM +0200, Georg Brandl wrote:
>> On 20.06.2011 02:42, senthil.kumaran wrote:
>> > summary:
>> >   Fix closes Issue12359 - Minor update to module import description.
>> > ...
>> > +When a module named :mod:`spam` is imported, the interpreter searches for a
>> > +file named :file:`spam.py` in the directory containing the input script (or
>> > +thecurrent directory), and then in the list of directories specified by the
>> 
>> By just adding "or the current directory", you've actually made this more
>> confusing: now the reader will wonder when it's the script directory and when
>> it's the current directory.
> 
> I added that statement in the bracket, after looking at another
> instance in the next para which had this mention. I think, the point
> here is that the reader would understand, where the import is looking
> for the module based on the context.  Fine with removing this sentence
> ("or the current directory"), if statement explains the idea better
> without it.

Thanks!

Georg



From opensourcesurvey at cs.ua.edu  Mon Jun 20 21:45:27 2011
From: opensourcesurvey at cs.ua.edu (Jeffrey Carver)
Date: Mon, 20 Jun 2011 14:45:27 -0500
Subject: [Python-Dev] REMINDER: Participation Requested: Survey about
	Open-Source Software Development
Message-ID: <00f501cc2f82$9ada3f30$d08ebd90$@cs.ua.edu>

Hi,

Apologies for any inconvenience and thank you to those who have already
completed the survey. We will keep the survey open for another couple of
weeks. But, we do hope you will consider responding to the email request
below (sent 2 weeks ago).

Thanks,

Dr. Jeffrey Carver
Assistant Professor
University of Alabama
(v) 205-348-9829  (f) 205-348-0219
http://www.cs.ua.edu/~carver

-----Original Message-----
From: Jeffrey Carver [mailto:opensourcesurvey at cs.ua.edu] 
Sent: Monday, June 13, 2011 11:46 AM
To: 'python-dev at python.org'
Subject: Participation Requested: Survey about Open-Source Software
Development

Hi,

Drs. Jeffrey Carver, Rosanna Guadagno, Debra McCallum, and Mr. Amiangshu
Bosu,  University of Alabama, and Dr. Lorin Hochstein, University of
Southern California, are conducting a survey of open-source software
developers. This survey seeks to understand how developers on distributed,
virtual teams, like open-source projects, interact with each other to
accomplish their tasks. You must be at least 19 years of age to complete the
survey. The survey should take approximately 15 minutes to complete.

If you are actively participating as a developer, please consider completing
our survey.
 
Here is the link to the survey:   http://goo.gl/HQnux

We apologize for inconvenience and if you receive multiple copies of this
email. This survey has been approved by The University of Alabama IRB board.

Thanks,

Dr. Jeffrey Carver
Assistant Professor
University of Alabama
(v) 205-348-9829  (f) 205-348-0219
http://www.cs.ua.edu/~carver



From guido at python.org  Tue Jun 21 01:40:02 2011
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Jun 2011 16:40:02 -0700
Subject: [Python-Dev] Parser/intrcheck.c
In-Reply-To: <20110616002140.77861fdd@pitrou.net>
References: <20110616002140.77861fdd@pitrou.net>
Message-ID: <BANLkTineP-L9Wfq3gVPUVKYeCqnsxETL7Q@mail.gmail.com>

I think it's safe to remove it. The last reference to it I found was
in the 2.0 release, where there is a Parser/Makefile (generated from
Parser/Makefile.in) which contains the following gem:

# This target is used by the master Makefile to add the objects to the library
add2lib:        $(OBJS)
                $(AR) cr $(LIBRARY) $(AROBJS)
                if test ! -f ../Modules/hassignal; \
                then echo adding intrcheck.o; $(AR) r $(LIBRARY) intrcheck.o; \
                else echo leaving intrcheck.o out; fi
                touch add2lib

This Makefile.in was deleted in http://svn.python.org/view?view=revision

So I think you're fine killing that file.

--Guido

On Wed, Jun 15, 2011 at 3:21 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hello,
>
> I may be missing something, but I'm wondering whether
> Parser/intrcheck.c is still used anywhere.
>
> It's only mentioned in some comments:
>
> $ grep -r intrcheck.c *
> Modules/signalmodule.c:1197:/* Replacements for intrcheck.c functionality
> PC/os2vacpp/makefile.omk:217: ?# intrcheck.c ? ? -- Not Referenced by Anyone (?)
> Python/sigcheck.c:3: ? interrupt occurs. ?It can't be in the intrcheck.c file since that
>
> And if I remove it and "make clean", I can still rebuild successfully.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)

From guido at python.org  Tue Jun 21 01:40:27 2011
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Jun 2011 16:40:27 -0700
Subject: [Python-Dev] Parser/intrcheck.c
In-Reply-To: <BANLkTineP-L9Wfq3gVPUVKYeCqnsxETL7Q@mail.gmail.com>
References: <20110616002140.77861fdd@pitrou.net>
	<BANLkTineP-L9Wfq3gVPUVKYeCqnsxETL7Q@mail.gmail.com>
Message-ID: <BANLkTinb0QdOmXg3we1-U8kGpiGTsAYN_A@mail.gmail.com>

On Mon, Jun 20, 2011 at 4:40 PM, Guido van Rossum <guido at python.org> wrote:
> I think it's safe to remove it. The last reference to it I found was
> in the 2.0 release, where there is a Parser/Makefile (generated from
> Parser/Makefile.in) which contains the following gem:
>
> # This target is used by the master Makefile to add the objects to the library
> add2lib: ? ? ? ?$(OBJS)
> ? ? ? ? ? ? ? ?$(AR) cr $(LIBRARY) $(AROBJS)
> ? ? ? ? ? ? ? ?if test ! -f ../Modules/hassignal; \
> ? ? ? ? ? ? ? ?then echo adding intrcheck.o; $(AR) r $(LIBRARY) intrcheck.o; \
> ? ? ? ? ? ? ? ?else echo leaving intrcheck.o out; fi
> ? ? ? ? ? ? ? ?touch add2lib
>
> This Makefile.in was deleted in http://svn.python.org/view?view=revision

http://svn.python.org/view?view=revision&revision=19308

> So I think you're fine killing that file.
>
> --Guido
>
> On Wed, Jun 15, 2011 at 3:21 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> Hello,
>>
>> I may be missing something, but I'm wondering whether
>> Parser/intrcheck.c is still used anywhere.
>>
>> It's only mentioned in some comments:
>>
>> $ grep -r intrcheck.c *
>> Modules/signalmodule.c:1197:/* Replacements for intrcheck.c functionality
>> PC/os2vacpp/makefile.omk:217: ?# intrcheck.c ? ? -- Not Referenced by Anyone (?)
>> Python/sigcheck.c:3: ? interrupt occurs. ?It can't be in the intrcheck.c file since that
>>
>> And if I remove it and "make clean", I can still rebuild successfully.
>>
>> Regards
>>
>> Antoine.
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>



-- 
--Guido van Rossum (python.org/~guido)

From dickinsm at gmail.com  Tue Jun 21 10:53:46 2011
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 21 Jun 2011 10:53:46 +0200
Subject: [Python-Dev] Language summit writeup anyone?
In-Reply-To: <20110620133131.553e4c87@msiwind>
References: <BANLkTinwD=9hK0N=q0vxf9d-i9ht1xOiOQ@mail.gmail.com>
	<20110620133131.553e4c87@msiwind>
Message-ID: <BANLkTi=beba7LKzm8r6ubDnvb2xeyFZ=TA@mail.gmail.com>

On Mon, Jun 20, 2011 at 1:31 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Maciej Fijalkowski <fijall at gmail.com> a ?crit :
>>
>> Unfortunately I'm missing Europython (and language summit) this year.
>> Did anyone do a writeup on what was discussed?
>
> Mark Dickinson has been taking notes, but since there only a few of us
> (roughly 10 attendants), it was mostly casual and friendly chatting :)

Hi guys,

I'm working on it.  I'm soliciting feedback on a draft from those who
were present at the meeting (if you *were* present at the meeting and
didn't receive the draft writeup, please shout!).  Once I've
established that I haven't grossly misrepresented anyone, I'll send
the writeup to python-dev.

Mark

From ziade.tarek at gmail.com  Tue Jun 21 13:42:05 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Jun 2011 13:42:05 +0200
Subject: [Python-Dev] packaging backport
Message-ID: <BANLkTikHZobVO6V0buJZFHyUpMKBidAmEQ@mail.gmail.com>

Hello

I am starting the backport of Packaging into a standalone version
named Distutils2 -- this is very important to speed up the development
of packaging itself since it'll give of more feedback from 2.x
projects

To do this I need to create a script that does a mass renaming of
'packaging' into 'distutils2', then I can start to play with 3to2 and
...3.xto3.x :) .

I tried to script rope but the py3k version seem not compatible with
our current AST -- and it's a bit overkill for this task I guess.

Before I start to write my own refactoring tool, I was wondering if
anyone here had some experience in this, and could give me some hints.


Thanks
Tarek
-- 
Tarek Ziad? | http://ziade.org

From ncoghlan at gmail.com  Tue Jun 21 14:44:43 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 21 Jun 2011 22:44:43 +1000
Subject: [Python-Dev] packaging backport
In-Reply-To: <BANLkTikHZobVO6V0buJZFHyUpMKBidAmEQ@mail.gmail.com>
References: <BANLkTikHZobVO6V0buJZFHyUpMKBidAmEQ@mail.gmail.com>
Message-ID: <BANLkTimspF2_dJfOHzHSepJqiboyTRT-yw@mail.gmail.com>

2to3 or Brett's mnfy are likely to be reasonable starting points.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110621/19de97ec/attachment.html>

From rdmurray at bitdance.com  Tue Jun 21 15:12:38 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 21 Jun 2011 09:12:38 -0400
Subject: [Python-Dev] packaging backport
In-Reply-To: <BANLkTikHZobVO6V0buJZFHyUpMKBidAmEQ@mail.gmail.com>
References: <BANLkTikHZobVO6V0buJZFHyUpMKBidAmEQ@mail.gmail.com>
Message-ID: <20110621131238.B9DA72506D8@webabinitio.net>

On Tue, 21 Jun 2011 13:42:05 +0200, =?ISO-8859-1?Q?Tarek_Ziad=E9?= <ziade.tarek at gmail.com> wrote:
> Before I start to write my own refactoring tool, I was wondering if
> anyone here had some experience in this, and could give me some hints.

Coul you could just write a 3to2 fixer?  I don't know how hard it is to
run just a selected set of fixers (so that you could use it to generate
python3 code), but it seems to me that renaming modules is something
that 3to2 (and 2to3, of course) should be good at.

--
R. David Murray           http://www.bitdance.com

From ziade.tarek at gmail.com  Tue Jun 21 15:24:26 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Jun 2011 15:24:26 +0200
Subject: [Python-Dev] packaging backport
In-Reply-To: <BANLkTimspF2_dJfOHzHSepJqiboyTRT-yw@mail.gmail.com>
References: <BANLkTikHZobVO6V0buJZFHyUpMKBidAmEQ@mail.gmail.com>
	<BANLkTimspF2_dJfOHzHSepJqiboyTRT-yw@mail.gmail.com>
Message-ID: <BANLkTikmU6B==DSZwrNoCyoszXFek_YHcg@mail.gmail.com>

On Tue, Jun 21, 2011 at 2:44 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 2to3 or Brett's mnfy are likely to be reasonable starting points.

Will look at mnfy, thanks

-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue Jun 21 15:27:53 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Jun 2011 15:27:53 +0200
Subject: [Python-Dev] packaging backport
In-Reply-To: <20110621131238.B9DA72506D8@webabinitio.net>
References: <BANLkTikHZobVO6V0buJZFHyUpMKBidAmEQ@mail.gmail.com>
	<20110621131238.B9DA72506D8@webabinitio.net>
Message-ID: <BANLkTinwZx6z6bAf93Mc-btR_m1A+ajK3w@mail.gmail.com>

On Tue, Jun 21, 2011 at 3:12 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Tue, 21 Jun 2011 13:42:05 +0200, =?ISO-8859-1?Q?Tarek_Ziad=E9?= <ziade.tarek at gmail.com> wrote:
>> Before I start to write my own refactoring tool, I was wondering if
>> anyone here had some experience in this, and could give me some hints.
>
> Coul you could just write a 3to2 fixer? ?I don't know how hard it is to
> run just a selected set of fixers (so that you could use it to generate
> python3 code), but it seems to me that renaming modules is something
> that 3to2 (and 2to3, of course) should be good at.

The one thing rope is good at is to find where a given variable name
is used, and rename all occurrences recursively. So basically, when
you rename an import, it renames all the code that uses it.

I don't really know how 2to3/3to2 work but I assumed that it does not
do this, but simply give you a hook for every visited node. IOW that
looking for dependencies is to be done

Cheers
Tarek

>
> --
> R. David Murray ? ? ? ? ? http://www.bitdance.com
>



-- 
Tarek Ziad? | http://ziade.org

From ziade.tarek at gmail.com  Tue Jun 21 15:43:09 2011
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 21 Jun 2011 15:43:09 +0200
Subject: [Python-Dev] packaging backport
In-Reply-To: <BANLkTimh6ic=agyGsHoYWJc=5Vj=y+t2oA@mail.gmail.com>
References: <BANLkTikHZobVO6V0buJZFHyUpMKBidAmEQ@mail.gmail.com>
	<20110621131238.B9DA72506D8@webabinitio.net>
	<BANLkTinwZx6z6bAf93Mc-btR_m1A+ajK3w@mail.gmail.com>
	<BANLkTimh6ic=agyGsHoYWJc=5Vj=y+t2oA@mail.gmail.com>
Message-ID: <BANLkTik1--N=etaaYn0qeKNW4d2g0evbOQ@mail.gmail.com>

On Tue, Jun 21, 2011 at 3:37 PM, Joe Amenta <amentajo at msu.edu> wrote:
...
>
> Yes, 2to3/3to2 does not do anything fancy like that. ?The tools are purely
> concerned with syntax, whereas renaming imports is semantic. ?The good news
> is that this line:
> import packaging as _myPackaging
> can be replaced by:
> import distutils2 as _myPackaging
> and code that uses _myPackaging will work. ?I've attached a fixer that can
> go into your lib3to2/fixes folder. ?You should also edit fix_imports.py and
> add the line:
> "packaging" : "distutils2",
> to the MAPPING dictionary near the top, then you can run 3to2 like this:
> 3to2 -fpackaging -fimports files_to_fix.py
> (-w option to write changes to the files modified)
> Hope this helps,

It does, thanks a lot !


Cheers
Tarek

-- 
Tarek Ziad? | http://ziade.org

From amentajo at msu.edu  Tue Jun 21 15:37:05 2011
From: amentajo at msu.edu (Joe Amenta)
Date: Tue, 21 Jun 2011 09:37:05 -0400
Subject: [Python-Dev] packaging backport
In-Reply-To: <BANLkTinwZx6z6bAf93Mc-btR_m1A+ajK3w@mail.gmail.com>
References: <BANLkTikHZobVO6V0buJZFHyUpMKBidAmEQ@mail.gmail.com>
	<20110621131238.B9DA72506D8@webabinitio.net>
	<BANLkTinwZx6z6bAf93Mc-btR_m1A+ajK3w@mail.gmail.com>
Message-ID: <BANLkTimh6ic=agyGsHoYWJc=5Vj=y+t2oA@mail.gmail.com>

On Tue, Jun 21, 2011 at 9:27 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> On Tue, Jun 21, 2011 at 3:12 PM, R. David Murray <rdmurray at bitdance.com>
> wrote:
> > On Tue, 21 Jun 2011 13:42:05 +0200, =?ISO-8859-1?Q?Tarek_Ziad=E9?= <
> ziade.tarek at gmail.com> wrote:
> >> Before I start to write my own refactoring tool, I was wondering if
> >> anyone here had some experience in this, and could give me some hints.
> >
> > Coul you could just write a 3to2 fixer?  I don't know how hard it is to
> > run just a selected set of fixers (so that you could use it to generate
> > python3 code), but it seems to me that renaming modules is something
> > that 3to2 (and 2to3, of course) should be good at.
>
> The one thing rope is good at is to find where a given variable name
> is used, and rename all occurrences recursively. So basically, when
> you rename an import, it renames all the code that uses it.
>
> I don't really know how 2to3/3to2 work but I assumed that it does not
> do this, but simply give you a hook for every visited node. IOW that
> looking for dependencies is to be done
>
> Cheers
> Tarek
>
> >
> > --
> > R. David Murray           http://www.bitdance.com
> >
>
>
>
> --
> Tarek Ziad? | http://ziade.org
>

Yes, 2to3/3to2 does not do anything fancy like that.  The tools are purely
concerned with syntax, whereas renaming imports is semantic.  The good news
is that this line:

import packaging as _myPackaging

can be replaced by:

import distutils2 as _myPackaging

and code that uses _myPackaging will work.  I've attached a fixer that can
go into your lib3to2/fixes folder.  You should also edit fix_imports.py and
add the line:

"packaging" : "distutils2",

to the MAPPING dictionary near the top, then you can run 3to2 like this:

3to2 -fpackaging -fimports files_to_fix.py

(-w option to write changes to the files modified)

Hope this helps,
--Joe Amenta
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110621/665d043b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_packaging.py
Type: application/octet-stream
Size: 358 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110621/665d043b/attachment.obj>

From vinay_sajip at yahoo.co.uk  Wed Jun 22 02:06:39 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 22 Jun 2011 00:06:39 +0000 (UTC)
Subject: [Python-Dev] In-Python virtualisation and packaging
References: <loom.20110613T132530-565@post.gmane.org>
	<4DF5FAC8.4090606@voidspace.org.uk>
	<BANLkTikQLdqemCZpiZXtqXtZjpSM1ED99g@mail.gmail.com>
Message-ID: <loom.20110622T014826-286@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> How well does the regression test suite cope when run inside such a
> virtualised environment?

I followed this up, and three tests fail: test_lib2to3, test_packaging and
test_sysconfig. These are errors which show up on the default branch too [1][2];
full results are at [3]. I've been keeping the pythonv branch synchronised with
default - these results appear to be quite stable/repeatable (old versions of
the results are available in the gist at [3]).

I did another test: in a pythonv-created environment, I tested pythonv/pysetup3
by trying to install all PyPI packages with a Python 3 trove classifier, where a
source distribution can be found. This smoke test shows a total of 398 such
packages, 310 of which were installed in the environment without errors. That's
78% - not too bad for this early stage in the game. The details of the failing
88 packages are at [4], and some of these are pysetup3 issues but a fair few are
bugs in the packages themselves (e.g. SyntaxErrors in setup.py, or missing
readme files that setup.py expects to be there) or missing dependencies like
boost.python or similar C-level dependencies. These tests were done with a
patched version of Distribute which uses sys.site_prefix is available, falling
back to sys.prefix when not (so the Distribute change is backward compatible).

Regards,

Vinay Sajip

[1] http://bugs.python.org/issue12331
[2] http://bugs.python.org/issue9100
[3] https://gist.github.com/1022705
[4] http://gist.github.com/1037662


From techtonik at gmail.com  Wed Jun 22 14:47:01 2011
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 22 Jun 2011 15:47:01 +0300
Subject: [Python-Dev] Is there any fun with benchmarks
Message-ID: <BANLkTinArA6LeOoS6jsp7iypxgqLTd2F6w@mail.gmail.com>

I run across a snippet in SCons.Util (don't worry, I've double-checked
To: field) that claims it is faster than os.path.splitext() while
basically doing the same thing.

def splitext(path):
    "Same as os.path.splitext() but faster."
    sep = rightmost_separator(path, os.sep)
    dot = path.rfind('.')
    # An ext is only real if it has at least one non-digit char
    if dot > sep and not containsOnly(path[dot:], "0123456789."):
        return path[:dot],path[dot:]
    else:
        return path,""

I wonder if upcoming speed.python.org has any means to validate these
claims for different Python releases?
Is there any place where I can upload my two to compare performance?
Are there any instructions how to create such snippets and add/enhance
dataset for them?
Any plans or opinions if that will be useful or not?
--
anatoly t.

From p.f.moore at gmail.com  Wed Jun 22 15:08:15 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 22 Jun 2011 14:08:15 +0100
Subject: [Python-Dev] Is there any fun with benchmarks
In-Reply-To: <BANLkTinArA6LeOoS6jsp7iypxgqLTd2F6w@mail.gmail.com>
References: <BANLkTinArA6LeOoS6jsp7iypxgqLTd2F6w@mail.gmail.com>
Message-ID: <BANLkTinZ7LFqPXTRkEFy_RRwaTrmbgpWRQ@mail.gmail.com>

On 22 June 2011 13:47, anatoly techtonik <techtonik at gmail.com> wrote:
> I run across a snippet in SCons.Util (don't worry, I've double-checked
> To: field) that claims it is faster than os.path.splitext() while
> basically doing the same thing.

Actually, it doesn't do the same thing. Doesn't handle files like
.profile properly. Also, this one seems to treat numerics differently.
So I'm not sure what you're trying to prove in a comparison...?

Paul.

From ncoghlan at gmail.com  Wed Jun 22 15:24:38 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Jun 2011 23:24:38 +1000
Subject: [Python-Dev] Is there any fun with benchmarks
In-Reply-To: <BANLkTinArA6LeOoS6jsp7iypxgqLTd2F6w@mail.gmail.com>
References: <BANLkTinArA6LeOoS6jsp7iypxgqLTd2F6w@mail.gmail.com>
Message-ID: <BANLkTimJzPF4SBQfk4F_FrH0GZ6ifUXj6Q@mail.gmail.com>

On Wed, Jun 22, 2011 at 10:47 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> I wonder if upcoming speed.python.org has any means to validate these
> claims for different Python releases?
> Is there any place where I can upload my two to compare performance?
> Are there any instructions how to create such snippets and add/enhance
> dataset for them?
> Any plans or opinions if that will be useful or not?

The timeit module handles microbenchmarks on short snippets without
any real problems. speed.python.org is about *macro* benchmarks -
getting a feel for the overall interpreter performance under a variety
of real world workflows.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From fijall at gmail.com  Wed Jun 22 17:20:53 2011
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 22 Jun 2011 17:20:53 +0200
Subject: [Python-Dev] Is there any fun with benchmarks
In-Reply-To: <BANLkTimJzPF4SBQfk4F_FrH0GZ6ifUXj6Q@mail.gmail.com>
References: <BANLkTinArA6LeOoS6jsp7iypxgqLTd2F6w@mail.gmail.com>
	<BANLkTimJzPF4SBQfk4F_FrH0GZ6ifUXj6Q@mail.gmail.com>
Message-ID: <BANLkTimKqmrBBbn1CJS5-9JRH-xkhVcGSg@mail.gmail.com>

On Wed, Jun 22, 2011 at 3:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Jun 22, 2011 at 10:47 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>> I wonder if upcoming speed.python.org has any means to validate these
>> claims for different Python releases?
>> Is there any place where I can upload my two to compare performance?
>> Are there any instructions how to create such snippets and add/enhance
>> dataset for them?
>> Any plans or opinions if that will be useful or not?
>
> The timeit module handles microbenchmarks on short snippets without
> any real problems. speed.python.org is about *macro* benchmarks -
> getting a feel for the overall interpreter performance under a variety
> of real world workflows.
>
> Cheers,
> Nick.
>

I think the question that timeit doesn't answer and speed potentially
can (I don't know if it should, but that's a matter of opinion) is how
those numbers differ among various interpreters/OSes/versions. This is
something for what you need a special offloaded server support

From barry at python.org  Wed Jun 22 18:39:08 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 22 Jun 2011 12:39:08 -0400
Subject: [Python-Dev] PEP 382 sprint
Message-ID: <20110622123908.4b65c1d5@neurotica.colubris.lan>

Hi folks,

Yesterday, 6 Washington DC area Pythonistas met to work on PEP 382.  I wrote
up a summary based on my notes and blogged about it here:

http://www.wefearchange.org/2011/06/pep-382-sprint-summary.html

Hopefully, the other participants will correct my mistakes and fill in the
holes.   A few other things to mention:

 * I resurrected the import-sig in order to shepherd PEP 382 to landing in
   Python 3.3.  If you're at all interested in helping out, please join the
   mailing list:

   http://mail.python.org/mailman/admin/import-sig

* We created a wiki page to track our results, questions, and plan of action:

  http://wiki.python.org/moin/Pep382Sprint

I want to thank my fellow sprint participants for coming out and really doing
a great job working on this.  And I especially want to thank the PSF for
sponsoring our sprint.  What a great way to encourage Python developers to
meet and work on improving Python!

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110622/09e426a8/attachment.pgp>

From g.brandl at gmx.net  Wed Jun 22 21:40:57 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 22 Jun 2011 21:40:57 +0200
Subject: [Python-Dev] cpython: #1874: detect invalid multipart CTE and
 report it as a defect.
In-Reply-To: <E1QZRXP-0007fF-Mi@dinsdale.python.org>
References: <E1QZRXP-0007fF-Mi@dinsdale.python.org>
Message-ID: <ittgf9$svv$1@dough.gmane.org>

On 22.06.2011 19:48, r.david.murray wrote:
> http://hg.python.org/cpython/rev/5a2602939d5d
> changeset:   70925:5a2602939d5d
> user:        R David Murray <rdmurray at bitdance.com>
> date:        Wed Jun 22 13:47:53 2011 -0400
> summary:
>   #1874: detect invalid multipart CTE and report it as a defect.
> 
> files:
>   Lib/email/errors.py               |   3 +
>   Lib/email/feedparser.py           |   5 ++
>   Lib/test/test_email/test_email.py |  45 +++++++++++++++++++
>   Misc/NEWS                         |   3 +
>   4 files changed, 56 insertions(+), 0 deletions(-)
> 
> 
> diff --git a/Lib/email/errors.py b/Lib/email/errors.py
> --- a/Lib/email/errors.py
> +++ b/Lib/email/errors.py
> @@ -55,3 +55,6 @@
>  
>  class MultipartInvariantViolationDefect(MessageDefect):
>      """A message claimed to be a multipart but no subparts were found."""
> +
> +class InvalidMultipartContentTransferEncodingDefect(MessageDefect):
> +    """An invalid content transfer encoding was set on the multipart itself."""

Dear Mr. Murray,

thank you very much for competing in the PSU's Longest Class Name in the
Standard Library competition.  Unfortunately, your class name of 45 characters
has been surpassed by four other contestants.

However, it is my pleasure to inform you that your entry wins the consolation
prize for the Longest Exception Name!  A framed certificate and a PSU-branded
wooden keyboard will be delivered to you shortly.

Yours sincerely,
the PSU ministry of silly stats



From rdmurray at bitdance.com  Wed Jun 22 23:40:12 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 22 Jun 2011 17:40:12 -0400
Subject: [Python-Dev] cpython: #1874: detect invalid multipart CTE and
	report it as a defect.
In-Reply-To: <ittgf9$svv$1@dough.gmane.org>
References: <E1QZRXP-0007fF-Mi@dinsdale.python.org>
	<ittgf9$svv$1@dough.gmane.org>
Message-ID: <20110622214013.157E62506D2@webabinitio.net>

On Wed, 22 Jun 2011 21:40:57 +0200, Georg Brandl <g.brandl at gmx.net> wrote:
> On 22.06.2011 19:48, r.david.murray wrote:
> > diff --git a/Lib/email/errors.py b/Lib/email/errors.py
> > --- a/Lib/email/errors.py
> > +++ b/Lib/email/errors.py
> > @@ -55,3 +55,6 @@
> >  
> >  class MultipartInvariantViolationDefect(MessageDefect):
> >      """A message claimed to be a multipart but no subparts were found."""
> > +
> > +class InvalidMultipartContentTransferEncodingDefect(MessageDefect):
> > +    """An invalid content transfer encoding was set on the multipart itself."""
> 
> Dear Mr. Murray,
> 
> thank you very much for competing in the PSU's Longest Class Name in the
> Standard Library competition.  Unfortunately, your class name of 45 characters
> has been surpassed by four other contestants.
> 
> However, it is my pleasure to inform you that your entry wins the consolation
> prize for the Longest Exception Name!  A framed certificate and a PSU-branded
> wooden keyboard will be delivered to you shortly.

See, there are hidden benefits to following the existing coding
conventions of stdlib modules...

(I initially called it InvalidMultipartCTEDefect, but all of the other
names were spelled out, so....)

--
R. David Murray           http://www.bitdance.com

From dickinsm at gmail.com  Fri Jun 24 10:52:40 2011
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 24 Jun 2011 10:52:40 +0200
Subject: [Python-Dev] EuroPython Language Summit report
Message-ID: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>

EuroPython 2011 Language Summit
===============================

Here's a brief report on the EuroPython 2011 Language Summit, held on Sunday 19
June, 2011 in Florence.  It was a fairly small meeting, with a lot of informal
and loosely-focused discussion and few conclusions reached.  I've outlined the
topics that we discussed below.

Present:

    Antonio Cuni
    Mark Dickinson
    Larry Hastings (chair)
    Marc-Andr? Lemburg
    Ezio Melotti
    Antoine Pitrou
    Armin Ronacher
    Armin Rigo
    Mark Ramm

Topics covered
==============

Python 3 adoption
-----------------

This was a long and wide-ranging discussion on the general state of Python 3
adoption.  Some highlights:

  - Even amongst those present at the meeting, most are still using Python 2
    for everyday work.

  - pypy: no definitive plans yet for PyPy supporting Python 3.

  - The web community is still very much focused on Python 2.

  - There's ongoing work (or perhaps just discussion?) on being able to
    generate Python 3 documentation using Sphinx running on Python 3.  This
    would be particularly useful when using extensions like 'autodoc' for
    Python 3 code.

  - [Armin Ronacher] Python 3's Unicode support still has some dark areas.  One
    example: when opening a text file for reading and writing, the default
    encoding used depends on the platform and on various environment variables.

  - Regular expression support in Python 3 needs improvement (support for
    Unicode character classes is a particular need).  [Subtopic:  what needs
    to be done to get the new regex module into Python?  Should it replace
    the existing module?  What about backwards compatibility issues?]

  - 2to3:  It's still painful to upgrade a codebase from Python 2 to Python 3,
    and there doesn't seem to be a consensus on best practices yet.

There was a long discussion that I can't hope to do justice to on whether 2to3
should be part of the standard library or not.  (With side discussions on
distutils and packaging in Python 3.)  Armin Ronacher was one of the main
participants in this discussion (and IIUC, the main opponent of having 2to3 in
the standard library); Armin, do you want to summarize this?

Another major topic of discussion was on how to manage Python 2 and Python 3 in
plugin environments (e.g., Blender): is there some way that both Python 2 and
Python 3 plugins could be used within the same system?  This got a bit
technical; perhaps someone else at the meeting would like to elaborate on this?

State of Python 3 on the web
----------------------------

Brief discussion.  Summary: it's improving; better than before.  We have
CherryPy and WSGI.  There are issues in WSGI that are going to become more
apparent as more people move to Python 3 frameworks.

Discussion of open PEPS
-----------------------

PEP 3151: (Reworking the OS and IO exception hierarchy.)  Antoine summarized
    this; no real discussion here.

PEP 3118: (Not open, but ...)  Revising the buffer protocol.  The buffer
    protocol implementation is still buggy and incomplete.  (Mostly okay for 1d
    buffers, not even close for higher-dimensional work.)

PEP 397: Python launcher for Windows.  (This was actually discussed later,
    after lunch; apologies for messing with the timeline here.)  There was some
    confusion over the purpose of this PEP.  Armin Ronacher had some objections
    to the PEP in its current form, but it's not clear to me what those objects
    are, or whether they still exist.  Armin, care to elaborate?

PEP 0380: Syntax for delegating to a subgenerator.

PEP 3150: Statement local namespaces.

PEP 3152: Cofunctions.


For all three of the above PEPs, there was some feeling that additional syntax
changes to the language are unnecessary and make it harder to learn; where
possible, we should prefer using 3rd partly libraries.

Issue 12326 (Linux 3)
---------------------

>From the issue report by Charles-Fran?ois Natali::

    Linus recently decided that the next Linux kernel version would 3.0 [1].
    As a consequence, sys.platform (which is actually MACHDEP) will be 'linux3'
    on machines running Linux 3 kernels, and tests checking explicitely against
    'linux2' will either break and won't run.

There was significant discussion on how this should be solved.  While there's a
lot of discussion already on the tracker, the choice of solution may have
significant ramifications, so it seems a good idea that this be discussed more
widely on python-dev.

Major questions::

- what should be done with Python 2.7?  As it stands, as I understand it,
  sys.platform *will* become 'linux3' on Linux 3.x (it's a computed value
  somewhere) in Python 2.7.2, and since Python 2.7.2 is already out there we
  can't change that.  Some of the proposed solutions to the issue would
  involve sys.platform changing between Python 2.7.2 and Python 2.7.3; is
  such a change acceptable for a bugfix release?

- it's easy to fix up all uses of "== 'linux2' " in the standard library; is it
  okay to just let 3rd party code break here?

- What should be done about Lib/plat-linux2?


The rough consensus at the summit was that for Python 2.7 at least, the only
realistic solution seems to be to do nothing except document the problem, and
point people to the platform module.  I'm not sure if this answers the
Lib/plat-linux2 question.

Python 2.7
----------

How long will this be maintained?
Original decision was 5 years.


PyPI on EC2
-----------

There was some discussion several months ago about the possibility of hosting
PyPI on EC2.  Is this still something that should be considered?  PyPI seems to
have been much more problem free in recent times.


virtualenv in Python 3.3?
-------------------------

Apparently there was some discussion at the last PyCon about the possibility of
virtualenv going into Python 3.3.  As far as I know there's currently no open
tracker item or PEP for this.  Larry Hastings knows more here.

-- Update from Larry: """I'm supportive of putting it in in 3.3, and I'm
   sitting with Raymond Hettinger right now and he supports it too.  So I think
   if we get a working implementation it'll go in.  It's under heavy discussion
   in c.l.p-d so I gather it's moving forward.  Last I knew it was Carl Meyer
   pushing it forward, but Vinay Sanjip is the current standard-bearer.  I
   understand Windows support will be a bit tricky; I don't know if they have
   someone who's going to handle it for them."""

Python website
--------------

The Python website is painful to edit right now.

From victor.stinner at haypocalc.com  Fri Jun 24 13:18:44 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 24 Jun 2011 13:18:44 +0200
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
Message-ID: <1308914324.2342.7.camel@marge>

Le vendredi 24 juin 2011 ? 10:52 +0200, Mark Dickinson a ?crit :
>   - [Armin Ronacher] Python 3's Unicode support still has some dark areas.

What? Unicode support is perfect in Python 3!

>   One  example: when opening a text file for reading and writing, the default
>     encoding used depends on the platform and on various environment variables.

... oh, I agree. This choice is a big portability issue. Mac OS X, most
Linux distro, BSD systems use UTF-8 local encoding, whereas Windows use
legacy code pages like cp1252 (something like ISO-8859-1) or cp952
(shift jis). But sometimes, the locale is "C" (e.g. on our buildbots)
and programs start to fail with Unicode errors...

I see two options to improve the situation.


(1) hard way: change open() API to make encoding a mandatory argument.
Problem: it breaks compatibility with Python 3.0, 3.1 and 3.2 (ooops!);
the encoding argument is the 4th argument, you have to use a keyword or
choose a value for the buffering argument. I proposed to change open()
API in Python 3.1 to make all arguments -except the filename and the
mode- keyword-only arguments, but Guido rejected my idea:

"Remember, for 3.0 we're trying to get a release out of the door, not
cram in new features, no matter how small."

http://bugs.python.org/issue4121


(2) soft way: add a warning if the encoding is implicit (locale
encoding). I don't know what is the best warning type, and if it should
be always displayed, only once, or not by default. Even if it is hidden
by default, a careful developer will be able to use -Werror to fix
bugs... I suspect that most tests fail if open() raises an exception if
the encoding is not specified (e.g. see distutils/packaging issues about
the file encoding).

Victor


From ncoghlan at gmail.com  Fri Jun 24 14:50:05 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 24 Jun 2011 22:50:05 +1000
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
Message-ID: <BANLkTik_kPf+ufoy7brLk30j_Yh1oApb0g@mail.gmail.com>

On Fri, Jun 24, 2011 at 6:52 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> PEP 3118: (Not open, but ...) ?Revising the buffer protocol. ?The buffer
> ? ?protocol implementation is still buggy and incomplete. ?(Mostly okay for 1d
> ? ?buffers, not even close for higher-dimensional work.)

Issue 10181 is the place to start for anyone that wants to help with
this one! :)

Cheers,
Nick.

--
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Fri Jun 24 15:08:09 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 24 Jun 2011 13:08:09 +0000 (UTC)
Subject: [Python-Dev] EuroPython Language Summit report
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
Message-ID: <loom.20110624T150325-9@post.gmane.org>

Mark Dickinson <dickinsm <at> gmail.com> writes:

> virtualenv in Python 3.3?
> -------------------------
> 
> Apparently there was some discussion at the last PyCon about the possibility
> of virtualenv going into Python 3.3.  As far as I know there's currently no
> open tracker item or PEP for this.  Larry Hastings knows more here.
>
>  Update from Larry: """I'm supportive of putting it in in 3.3, and I'm
>  sitting with Raymond Hettinger right now and he supports it too.  So I
>  think if we get a working implementation it'll go in.  It's under heavy
>  discussion in c.l.p-d so I gather it's moving forward.  Last I knew it was
>  Carl Meyer pushing it forward, but Vinay Sajip is the current standard-
>  bearer.  I understand Windows support will be a bit tricky; I don't know if
>  they have someone who's going to handle it for them."""

Mark, thanks for the summary.

Re. a PEP for virtual environments in Python, Carl is working on the PEP. The
first draft by Carl with my initial comments is at [1].  There's still some
work to do before we can actually present it as a PEP we're happy with.

I'm pleased to report good progress with the proof of concept implementation.
There are some issues still with packaging which I'm working through with ?ric
Araujo, but I've gone ahead and committed changes on my branch to get things
working. It's a good exercising of the packaging functionality :-)

What I have at the moment is pythonv [2]:

A modified Python which supports virtual environments via changes in
Modules/getpath.c, PC/getpathp.c, Lib/site.py, Lib/sysconfig.py,
Lib/sysconfig.cfg and Lib/distutils/sysconfig.py.

These changes basically detect if you're running in a virtual environment by
looking for an env.cfg, and if found, fixing it up so sys.path is set as it
should be. In addition, sys.site_prefix and sys.site_exec_prefix are created -
if not in a virtual env, these have the same values as sys.prefix and
sys.exec_prefix. With just these changes, the basics of a virtual environment
are provided (in terms of isolation from other environments and the system
Python). However, in order to be useful, the packaging tools have to work with
sys.site_prefix and sys.site_exec_prefix, so changes have been made to
sysconfig, distutils and packaging to do this.

I'm presently working on a demonstration application which integrates the
above work with Doug Hellmann's virtualenvwrapper and Guillermo L?pez' port of
it to Powershell to get a nice basic tool that'll support virtual environments
with packaging as an easy-to-use CLI, as well as Distribute support while
people migrate over to packaging, on Windows as well as Linux.

I'm expecting to cover both the Linux and Windows aspects (though I won't say
no to help ;-) and working through packaging issues relating to improved
Windows support. The basic functionality is there now, both in Windows and
Linux - the focus of work is in the ease-of-use CLI stuff which is not
envisaged to be part of core Python, but nevertheless needs to be done to make
virtual envs as painless as possible.

BTW Of the 398 packages on PyPI having a Py3K trove classifier, I've tested
installing all of them into a pythonv virtual env with Distribute, and 310
install without errors. Of the remaining 88, some are due to missing
dependencies, others due to broken packages on PyPI.

BTW of the Python regression test suite passes all tests in a pythonv
virtualenv, other than those for which there already are tracker issues for
the default branch in  hg.python.org/cpython (test_lib2to3, test_packaging,
test_sysconfig). Full test results are at [3].

All in all, it's looking reasonably good, and we hope to report more progress
on the PEP and the proof of concept soon!

Regards,

Vinay Sajip

[1] http://goo.gl/6u0S0
[2] https://bitbucket.org/vinay.sajip/pythonv
[3] https://gist.github.com/1022705


From rdmurray at bitdance.com  Fri Jun 24 16:55:15 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 24 Jun 2011 10:55:15 -0400
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
Message-ID: <20110624145516.84DA32506EA@webabinitio.net>

On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinson <dickinsm at gmail.com> wrote:
> EuroPython 2011 Language Summit
> ===============================
[...] 
>     Unicode character classes is a particular need).  [Subtopic:  what needs
>     to be done to get the new regex module into Python?  Should it replace
>     the existing module?  What about backwards compatibility issues?]

I'm pretty sure regex has backward compatibility as a goal for just this
reason (so it can replace the current module).

> PEP 0380: Syntax for delegating to a subgenerator.
> 
> PEP 3150: Statement local namespaces.
> 
> PEP 3152: Cofunctions.
> 
> For all three of the above PEPs, there was some feeling that additional syntax
> changes to the language are unnecessary and make it harder to learn; where
> possible, we should prefer using 3rd partly libraries.

I disagree with this with respect to 380.  Haven't looked at the others.

> Python 2.7
> ----------
> 
> How long will this be maintained?
> Original decision was 5 years.

As I recall, the actual decision was "at *least* 5 years".  It's only
been one so far...was the discussion that five years was too short or
too long?

As the code bases continue to drift apart, and we fix the fixable bugs,
the number of patches going in to 2.7 will decrease over time, so I
don't think the burden of continuing to support it will be too heavy.
Currently about half of the non-feature-request issues (ie: bugs) in
the tracker are tagged 2.7.  I expect to see that percentage continue
to decrease over time.

--
R. David Murray           http://www.bitdance.com

From status at bugs.python.org  Fri Jun 24 18:07:25 2011
From: status at bugs.python.org (Python tracker)
Date: Fri, 24 Jun 2011 18:07:25 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20110624160725.00E5B1D41C@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2011-06-17 - 2011-06-24)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    2845 ( +2)
  closed 21335 (+43)
  total  24180 (+45)

Open issues with patches: 1235 


Issues opened (30)
==================

#3067: setlocale error message is confusing
http://bugs.python.org/issue3067  reopened by terry.reedy

#12354: packaging.pypi.simple docs use both client and crawler variabl
http://bugs.python.org/issue12354  opened by gruszczy

#12355: Crawler doesn't follow redirection
http://bugs.python.org/issue12355  opened by gruszczy

#12361: Memory Leak in File Logging
http://bugs.python.org/issue12361  opened by japerk

#12364: Timeout (1 hour) in test_concurrent_futures.tearDown() on spar
http://bugs.python.org/issue12364  opened by haypo

#12365: URLopener should support context manager protocol
http://bugs.python.org/issue12365  opened by mcjeff

#12366: packaging.pypi.dist should abstract download errors.
http://bugs.python.org/issue12366  opened by michael.mulich

#12368: packaging.pypi.simple.Crawler assumes external download links 
http://bugs.python.org/issue12368  opened by michael.mulich

#12372: semaphore errors on AIX 7.1
http://bugs.python.org/issue12372  opened by reshmi.george

#12374: Execution model should explain compile vs definition vs execut
http://bugs.python.org/issue12374  opened by ncoghlan

#12375: Add packages_root to sys.path for hooks
http://bugs.python.org/issue12375  opened by erik.bray

#12376: unittest.TextTestResult.__init__ breaks under complex __mro__
http://bugs.python.org/issue12376  opened by branker

#12377: Clean up use of packages_root/package_dir
http://bugs.python.org/issue12377  opened by erik.bray

#12378: smtplib.SMTP_SSL leaks socket connections on SSL error
http://bugs.python.org/issue12378  opened by joeshaw

#12379: build outside source fail in head
http://bugs.python.org/issue12379  opened by rpetrov

#12380: bytearray methods center, ljust, rjust don't accept a bytearra
http://bugs.python.org/issue12380  opened by py.user

#12381: bytearray methods count, find, index don't support None as in 
http://bugs.python.org/issue12381  opened by py.user

#12382: [msilib] Obscure exception message when trying to open a non-e
http://bugs.python.org/issue12382  opened by Robin.Jarry

#12384: difflib.SequenceMatcher and Match: code and doc bugs
http://bugs.python.org/issue12384  opened by terry.reedy

#12385: the help for bytearray.maketrans describes bytes.maketrans
http://bugs.python.org/issue12385  opened by py.user

#12386: packaging fails in install_distinfo when writing RESOURCES
http://bugs.python.org/issue12386  opened by vinay.sajip

#12387: IDLE save keyboard shortcut problem
http://bugs.python.org/issue12387  opened by Jacob.VB

#12388: cannot specify recursive extra_files in packaging setup.cfg
http://bugs.python.org/issue12388  opened by vinay.sajip

#12391: packaging install fails to clean up temp files
http://bugs.python.org/issue12391  opened by vinay.sajip

#12392: pthread_kill() doesn't work on the main thread on FreeBSD6
http://bugs.python.org/issue12392  opened by haypo

#12393: Packaging should provide support for extensible categories
http://bugs.python.org/issue12393  opened by vinay.sajip

#12394: packaging: generate scripts from callable (dotted paths)
http://bugs.python.org/issue12394  opened by vinay.sajip

#12395: packaging remove fails under Windows
http://bugs.python.org/issue12395  opened by vinay.sajip

#12397: re match object methods have no docstrings
http://bugs.python.org/issue12397  opened by nedbat

#12398: Sending binary data with a POST request in httplib can cause U
http://bugs.python.org/issue12398  opened by sorin



Most recent 15 issues with no replies (15)
==========================================

#12398: Sending binary data with a POST request in httplib can cause U
http://bugs.python.org/issue12398

#12397: re match object methods have no docstrings
http://bugs.python.org/issue12397

#12391: packaging install fails to clean up temp files
http://bugs.python.org/issue12391

#12376: unittest.TextTestResult.__init__ breaks under complex __mro__
http://bugs.python.org/issue12376

#12347: add an "extras" in packaging.pypi.simple.Crawler
http://bugs.python.org/issue12347

#12336: tkinter.test.test_ttk.test_widgets.test_select() failure on Fr
http://bugs.python.org/issue12336

#12323: ElementPath 1.3 expressions
http://bugs.python.org/issue12323

#12322: ElementPath 1.3 expressions documentation
http://bugs.python.org/issue12322

#12317: inspect.getabsfile() is not documented
http://bugs.python.org/issue12317

#12297: Clarifications to atexit.register and unregister doc
http://bugs.python.org/issue12297

#12295: Fix ResourceWarning in turtledemo help window
http://bugs.python.org/issue12295

#12294: multiprocessing.Pool: Need a way to find out if work are finis
http://bugs.python.org/issue12294

#12288: Python 2.7.1 tkSimpleDialog initialvalue
http://bugs.python.org/issue12288

#12259: Test and document which compilers can be created on which plat
http://bugs.python.org/issue12259

#12258: Clean up bytes I/O in get_compiler_versions
http://bugs.python.org/issue12258



Most recent 15 issues waiting for review (15)
=============================================

#12392: pthread_kill() doesn't work on the main thread on FreeBSD6
http://bugs.python.org/issue12392

#12372: semaphore errors on AIX 7.1
http://bugs.python.org/issue12372

#12354: packaging.pypi.simple docs use both client and crawler variabl
http://bugs.python.org/issue12354

#12353: argparse cannot handle empty arguments
http://bugs.python.org/issue12353

#12350: Improve stat_result.st_blocks and st_blksize documentation
http://bugs.python.org/issue12350

#12344: Add **kwargs to get_reinitialized_command
http://bugs.python.org/issue12344

#12341: Some additions to .hgignore
http://bugs.python.org/issue12341

#12328: multiprocessing's overlapped PipeConnection on Windows
http://bugs.python.org/issue12328

#12308: Add link to PEP 0 for topical index in wiki
http://bugs.python.org/issue12308

#12306: zlib: Expose zlibVersion to query runtime version of zlib
http://bugs.python.org/issue12306

#12304: expose signalfd(2) in the signal module
http://bugs.python.org/issue12304

#12303: expose sigwaitinfo() and sigtimedwait() in the signal module
http://bugs.python.org/issue12303

#12297: Clarifications to atexit.register and unregister doc
http://bugs.python.org/issue12297

#12296: Minor clarification in devguide
http://bugs.python.org/issue12296

#12295: Fix ResourceWarning in turtledemo help window
http://bugs.python.org/issue12295



Top 10 most discussed issues (10)
=================================

#12291: file written using marshal in 3.2 can be read by 2.7, but not 
http://bugs.python.org/issue12291  14 msgs

#12394: packaging: generate scripts from callable (dotted paths)
http://bugs.python.org/issue12394  14 msgs

#12326: Linux 3: tests should avoid using sys.platform == 'linux2'
http://bugs.python.org/issue12326  11 msgs

#4470: smtplib SMTP_SSL not working.
http://bugs.python.org/issue4470   8 msgs

#12255: A few changes to .*ignore
http://bugs.python.org/issue12255   8 msgs

#2202: urllib2 fails against IIS 6.0 (No support for MD5-sess auth)
http://bugs.python.org/issue2202   7 msgs

#3067: setlocale error message is confusing
http://bugs.python.org/issue3067   7 msgs

#12353: argparse cannot handle empty arguments
http://bugs.python.org/issue12353   7 msgs

#10206: python program starting with unmatched quote spews spaces to s
http://bugs.python.org/issue10206   6 msgs

#11812: transient socket failure to connect to 'localhost'
http://bugs.python.org/issue11812   6 msgs



Issues closed (41)
==================

#1874: email parser does not register a defect for invalid Content-Tr
http://bugs.python.org/issue1874  closed by r.david.murray

#4015: Make installed scripts executable on windows
http://bugs.python.org/issue4015  closed by eric.araujo

#5905: strptime fails in non-UTF locale
http://bugs.python.org/issue5905  closed by haypo

#6697: Check that _PyUnicode_AsString() result is not NULL
http://bugs.python.org/issue6697  closed by haypo

#6734: Imap lib implicit conversion from bytes to string
http://bugs.python.org/issue6734  closed by r.david.murray

#6771: Curses.wrapper: documentation/implementation error
http://bugs.python.org/issue6771  closed by r.david.murray

#9921: os.path.join('x','') behavior
http://bugs.python.org/issue9921  closed by r.david.murray

#10354: tempfile.template is broken
http://bugs.python.org/issue10354  closed by r.david.murray

#10636: subprocess module has race	condition with SIGCHLD	handlers
http://bugs.python.org/issue10636  closed by rosslagerwall

#11637: Add cwd to sys.path for hooks
http://bugs.python.org/issue11637  closed by eric.araujo

#11690: Devguide: Add "communication" FAQ
http://bugs.python.org/issue11690  closed by ncoghlan

#11700: mailbox.py proxy close method cannot be called multiple times
http://bugs.python.org/issue11700  closed by r.david.murray

#11781: test/test_email directory does not get installed by 'make inst
http://bugs.python.org/issue11781  closed by r.david.murray

#11795: Better core dev guidelines for committing submitted patches
http://bugs.python.org/issue11795  closed by ncoghlan

#12090: 3.2: build --without-threads fails
http://bugs.python.org/issue12090  closed by georg.brandl

#12223: Datamodel documentation page: 'operator' where 'operand' shoul
http://bugs.python.org/issue12223  closed by eli.bendersky

#12261: urllib.parse docs still refer to urlparse
http://bugs.python.org/issue12261  closed by python-dev

#12278: Core mentorship mention in the devguide
http://bugs.python.org/issue12278  closed by python-dev

#12285: Unexpected behavior for 0 or negative processes in multiproces
http://bugs.python.org/issue12285  closed by haypo

#12289: http.server.CGIHTTPRequestHandler doesn't check if a Python sc
http://bugs.python.org/issue12289  closed by python-dev

#12313: make install misses test dirs for packaging and email modules
http://bugs.python.org/issue12313  closed by vinay.sajip

#12315: Improve http.client.HTTPResponse.read documentation
http://bugs.python.org/issue12315  closed by python-dev

#12337: Need real TextIOWrapper for stdin/stdout
http://bugs.python.org/issue12337  closed by haypo

#12356: more argument error improving
http://bugs.python.org/issue12356  closed by python-dev

#12357: Python dist modifications for secure PyPI uploads
http://bugs.python.org/issue12357  closed by eric.araujo

#12358: validate server certificate when uploading packages to PyPI
http://bugs.python.org/issue12358  closed by eric.araujo

#12359: tutorial: Module search path description is incorrect
http://bugs.python.org/issue12359  closed by python-dev

#12360: Doc Typo
http://bugs.python.org/issue12360  closed by python-dev

#12362: General Windows stdout redirection not working
http://bugs.python.org/issue12362  closed by r.david.murray

#12363: test_signal.test_without_siginterrupt() sporadic failures on F
http://bugs.python.org/issue12363  closed by python-dev

#12367: select.error has no errno attribute
http://bugs.python.org/issue12367  closed by pitrou

#12369: Revised core mentorship section of help.rst
http://bugs.python.org/issue12369  closed by ncoghlan

#12370: Use of super overwrites use of __class__ in class namespace
http://bugs.python.org/issue12370  closed by python-dev

#12371: datetime.now() bug
http://bugs.python.org/issue12371  closed by haypo

#12373: Duplicate packets in Multicast Receiver
http://bugs.python.org/issue12373  closed by neologix

#12383: subprocess.Popen(..., env={}) fails to pass empty env.
http://bugs.python.org/issue12383  closed by python-dev

#12389: typo in urllib: missing space after dot at sentence end
http://bugs.python.org/issue12389  closed by r.david.murray

#12390: urllib.parse.urlencode encoding lists as strings
http://bugs.python.org/issue12390  closed by orsenthil

#12396: Delivery failed
http://bugs.python.org/issue12396  closed by eric.araujo

#976869: Stripping script extensions with distutils
http://bugs.python.org/issue976869  closed by eric.araujo

#870479: Scripts need platform-dependent handling
http://bugs.python.org/issue870479  closed by eric.araujo

From tjreedy at udel.edu  Fri Jun 24 22:30:06 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 24 Jun 2011 16:30:06 -0400
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <1308914324.2342.7.camel@marge>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<1308914324.2342.7.camel@marge>
Message-ID: <iu2s4f$785$1@dough.gmane.org>

On 6/24/2011 7:18 AM, Victor Stinner wrote:
> Le vendredi 24 juin 2011 ? 10:52 +0200, Mark Dickinson a ?crit :
>>    One  example: when opening a text file for reading and writing, the default
>>      encoding used depends on the platform and on various environment variables.
>
> ... oh, I agree. This choice is a big portability issue. Mac OS X, most
> Linux distro, BSD systems use UTF-8 local encoding, whereas Windows use
> legacy code pages like cp1252 (something like ISO-8859-1) or cp952
> (shift jis). But sometimes, the locale is "C" (e.g. on our buildbots)
> and programs start to fail with Unicode errors...
>
> I see two options to improve the situation.

The third is to make utf-8 the default. I believe this *is* the proper 
long term solution and both options are contrary to this.
I believe that this is what I want for myself even on Windows.
I believe utf-8 is the default or only storage by cross-platform
international programs (certainly the ones I use).

> (1) hard way: change open() API to make encoding a mandatory argument.

> (2) soft way: add a warning if the encoding is implicit (locale
> encoding).

(3) In 3.3, if the default is used and it is not utf-8, add a warning 
that the default will become utf-8 always in 3.4. Actually, I would like 
a PendingDeprecationWarning in 3.2.1 if possible.

-- 
Terry Jan Reedy



From v+python at g.nevcal.com  Fri Jun 24 22:45:53 2011
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 24 Jun 2011 13:45:53 -0700
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <iu2s4f$785$1@dough.gmane.org>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>	<1308914324.2342.7.camel@marge>
	<iu2s4f$785$1@dough.gmane.org>
Message-ID: <4E04F781.60001@g.nevcal.com>

On 6/24/2011 1:30 PM, Terry Reedy wrote:
>
> The third is to make utf-8 the default. I believe this *is* the proper 
> long term solution and both options are contrary to this. 

+1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110624/ec5f2cd5/attachment.html>

From victor.stinner at haypocalc.com  Fri Jun 24 23:08:18 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 24 Jun 2011 23:08:18 +0200
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <iu2s4f$785$1@dough.gmane.org>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<1308914324.2342.7.camel@marge>  <iu2s4f$785$1@dough.gmane.org>
Message-ID: <1308949698.3352.24.camel@marge>

Le vendredi 24 juin 2011 ? 16:30 -0400, Terry Reedy a ?crit :
> > I see two options to improve the situation.
> 
> The third is to make utf-8 the default. I believe this *is* the proper 
> long term solution and both options are contrary to this.

Oh yes, I also prefer this option, but I suspect that some people prefer
to not break backward compatibility.

Or should we consider this bad design choice as a bug?

The UTF-8 encoder (of Python 3) raises an error if the text contains a
surrogate character. The surrogatepass error handler should be used to
encode surrogages.

... The surrogateescape can be used to encode back undecodable bytes
(e.g. filename decoded by Python using the surrogateescape), but it is
not a good idea to write illegal byte sequences (most programs will fail
to open the file).

> I believe that this is what I want for myself even on Windows.

Can you open a UTF-8 files in all Windows program (and the text is
displayed correctly)? I remember that notepad.exe writes an evil UTF-8
BOM, but I don't know if it requires this BOM to detect the UTF-8
encoding.

Or do some program expect text files encoded to the ANSI code page?

If you want to write files in the ANSI code page, you can use
encoding="mbcs" (or use an explicit code page, like encoding="cp1252").

> (3) In 3.3, if the default is used and it is not utf-8, add a warning 
> that the default will become utf-8 always in 3.4. Actually, I would like 
> a PendingDeprecationWarning in 3.2.1 if possible.

I'm not sure that the "and it is not utf-8" condition is a good idea. If
you develop on Linux but your users are on Windows, you will not get the
warning (even with -Werror) nor your users (users don't want to see
warnings)... Or maybe an user using Windows and Linux will notice the
bug without the warning :-)

It doesn't mean that it is not possible to check your program: you can
change your locale encoding (e.g. use LANG=C).

At least, it will be possible to check test_distutils and test_packaging
using LANG=C and -Werror :-)

--

A fourth option is to use ASCII by default! Your program will work and
be portable until you write the first non-ASCII character... Oh wait, it
remembers me the Unicode nightmare of Python 2!

Victor


From ncoghlan at gmail.com  Sat Jun 25 02:46:39 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 25 Jun 2011 10:46:39 +1000
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <20110624145516.84DA32506EA@webabinitio.net>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
Message-ID: <BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>

On Sat, Jun 25, 2011 at 12:55 AM, R. David Murray <rdmurray at bitdance.com> wrote:

[quoting VM summit notes]
>> PEP 0380: Syntax for delegating to a subgenerator.
>>
>> PEP 3150: Statement local namespaces.
>>
>> PEP 3152: Cofunctions.
>>
>> For all three of the above PEPs, there was some feeling that additional syntax
>> changes to the language are unnecessary and make it harder to learn; where
>> possible, we should prefer using 3rd partly libraries.
>
> I disagree with this with respect to 380. ?Haven't looked at the others.

Indeed, PEP 380 is *really* hard to do properly without language
support. The language moratorium and lack of a Python 3 compatible
patch are the only reasons it didn't go into 3.2 (and there's a patch
porting the implementation to 3.3 up on bitbucket, although we've been
tinkering with the compiler so it is likely out of date at this
point).

I'm the author PEP 3150 and *I* think it's a highly questionable and
most likely terrible idea (hence the Deferred status). However, it's a
good place to gather the related discussions, since proposals in that
vein recur often on python-ideas.

PEP 3152 definitely fits into the "let third parties experiment"
bucket, though - once PEP 380 makes generators first class peers to
functions in their support for modularisation, then we need to let the
wider community play with the concept for a while before we embed
anything more into the core language or standard library.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From greg.ewing at canterbury.ac.nz  Sat Jun 25 13:27:25 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 25 Jun 2011 23:27:25 +1200
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>
Message-ID: <4E05C61D.9090402@canterbury.ac.nz>

Nick Coghlan wrote:

> Indeed, PEP 380 is *really* hard to do properly without language
> support. The language moratorium and lack of a Python 3 compatible
> patch

Pardon? My original patch was for 3.1.2.

-- 
Greg

From ncoghlan at gmail.com  Sat Jun 25 14:55:48 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 25 Jun 2011 22:55:48 +1000
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <4E05C61D.9090402@canterbury.ac.nz>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>
	<4E05C61D.9090402@canterbury.ac.nz>
Message-ID: <BANLkTim5UbEzyh=-vHQMTP5D2b+DqSZuZQ@mail.gmail.com>

On Sat, Jun 25, 2011 at 9:27 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>
>> Indeed, PEP 380 is *really* hard to do properly without language
>> support. The language moratorium and lack of a Python 3 compatible
>> patch
>
> Pardon? My original patch was for 3.1.2.

My mistake. We must have changed something else somewhere along the
line that broke your patch...

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From pje at telecommunity.com  Sat Jun 25 17:31:42 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 25 Jun 2011 11:31:42 -0400
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>
Message-ID: <20110625153157.4B1063A4093@sparrow.telecommunity.com>

At 10:46 AM 6/25/2011 +1000, Nick Coghlan wrote:
>Indeed, PEP 380 is *really* hard to do properly without language
>support.

No, it isn't.  You add a decorator, a 'from_' class, and a 'return_' 
function, and there you go.  (See my previous code sketches here in 
early PEP 380 discussions.)

Python frameworks have been doing variations of the same thing (with 
varying features and APIs) for at least 7 years now -- even on Python 
versions that lack decorators or the ability to return values from 
yield statements.

So the main benefit of a PEP for this functionality would be 
providing a common implementation/API - and that could be initially 
done in the stdlib, without any added syntax support.


From rdmurray at bitdance.com  Sat Jun 25 18:32:41 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 25 Jun 2011 12:32:41 -0400
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <20110625153157.4B1063A4093@sparrow.telecommunity.com>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>
	<20110625153157.4B1063A4093@sparrow.telecommunity.com>
Message-ID: <20110625163242.349EC2506ED@webabinitio.net>

On Sat, 25 Jun 2011 11:31:42 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
> At 10:46 AM 6/25/2011 +1000, Nick Coghlan wrote:
> >Indeed, PEP 380 is *really* hard to do properly without language
> >support.
> 
> No, it isn't.  You add a decorator, a 'from_' class, and a 'return_' 
> function, and there you go.  (See my previous code sketches here in 
> early PEP 380 discussions.)
> 
> Python frameworks have been doing variations of the same thing (with 
> varying features and APIs) for at least 7 years now -- even on Python 
> versions that lack decorators or the ability to return values from 
> yield statements.
> 
> So the main benefit of a PEP for this functionality would be 
> providing a common implementation/API - and that could be initially 
> done in the stdlib, without any added syntax support.

So your proposed code would allow me, when writing a generator in
my code, do something that would allow me to yield up all the
values from an arbitrary generator I'm calling, over which I have
no control (ie: I can't modify its code)?

--
R. David Murray           http://www.bitdance.com

From riscutiavlad at gmail.com  Sat Jun 25 18:33:04 2011
From: riscutiavlad at gmail.com (Vlad Riscutia)
Date: Sat, 25 Jun 2011 09:33:04 -0700
Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy
Message-ID: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>

I recently started looking at some ctypes issues. I dug a bit into
http://bugs.python.org/issue6069 and then I found
http://bugs.python.org/issue11920. They both boil down to the fact that
bitfield allocation is up to the compiler, which is different in GCC and
MSVC. Currently we have hard-coded allocation strategy based on paltform in
cfields.c:

> if (bitsize /* this is a bitfield request */

>        && *pfield_size /* we have a bitfield open */
> #ifdef MS_WIN32
>        /* MSVC, GCC with -mms-bitfields */
>        && dict->size * 8 == *pfield_size
> #else
>        /* GCC */
>        && dict->size * 8 <= *pfield_size
> #endif
>        && (*pbitofs + bitsize) <= *pfield_size) {
>        /* continue bit field */
>        fieldtype = CONT_BITFIELD;
> #ifndef MS_WIN32
>    } else if (bitsize /* this is a bitfield request */
>        && *pfield_size /* we have a bitfield open */
>        && dict->size * 8 >= *pfield_size
>        && (*pbitofs + bitsize) <= dict->size * 8) {
>        /* expand bit field */
>        fieldtype = EXPAND_BITFIELD;
> #endif

So when creating a bitfield structure, it's size can be different on Linux
vs Windows.

class MyStructure(ctypes.BigEndianStructure):
    _pack_      = 1    # aligned to 8 bits, not ctypes default of 32
    _fields_    = [
                   ("Data0",   ctypes.c_uint32, 32),
                   ("Data1",   ctypes.c_uint8, 3),
                   ("Data2",   ctypes.c_uint16, 12),
                  ]

sizeof for above structure is 6 on GCC build and 7 on MSVC build. This leads
to some confusion and issues, because we can't always interop correctly with
code compiled with different compiler than the one Python is compiled with
on the platform. Short term solution is to add a warning in the
documentation about this. Longer term though, I think it would be better to
add a property on the Structure class for configurable allocation strategy,
for example Native (default), GCC, MSVC and when allocating the bitfield,
use given strategy. Native would behave similar to what happens now, but we
would also allow GCC-style allocation on Windows for example and the ability
to extend this if we ever run into similar issues with other compilers. This
shouldn't be too difficult to implement, will be backwards compatible and it
would improve interop. I would like to hear some opinions on this.

Thank you,
Vlad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110625/7b4f6fea/attachment.html>

From guido at python.org  Sat Jun 25 18:44:36 2011
From: guido at python.org (Guido van Rossum)
Date: Sat, 25 Jun 2011 09:44:36 -0700
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <20110625163242.349EC2506ED@webabinitio.net>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>
	<20110625153157.4B1063A4093@sparrow.telecommunity.com>
	<20110625163242.349EC2506ED@webabinitio.net>
Message-ID: <BANLkTimHyGaDmZELVvjibH0r4eHOtZfDAg@mail.gmail.com>

On Sat, Jun 25, 2011 at 9:32 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Sat, 25 Jun 2011 11:31:42 -0400, "P.J. Eby" <pje at telecommunity.com> wrote:
>> At 10:46 AM 6/25/2011 +1000, Nick Coghlan wrote:
>> >Indeed, PEP 380 is *really* hard to do properly without language
>> >support.
>>
>> No, it isn't. ?You add a decorator, a 'from_' class, and a 'return_'
>> function, and there you go. ?(See my previous code sketches here in
>> early PEP 380 discussions.)
>>
>> Python frameworks have been doing variations of the same thing (with
>> varying features and APIs) for at least 7 years now -- even on Python
>> versions that lack decorators or the ability to return values from
>> yield statements.
>>
>> So the main benefit of a PEP for this functionality would be
>> providing a common implementation/API - and that could be initially
>> done in the stdlib, without any added syntax support.
>
> So your proposed code would allow me, when writing a generator in
> my code, do something that would allow me to yield up all the
> values from an arbitrary generator I'm calling, over which I have
> no control (ie: I can't modify its code)?

Let me cut this short. PEP 380 is pretty much approved. I know there
are a few details worth quibbling over, but they are not going to
jeopardize acceptance of the PEP. We are waiting for an implementation
in Python 3.3. In fact, I wouldn't mind at this point if someone took
their best effort at an implementation and checked it into the 3.3
branch, and we can do the quibbling over the details while we have a
working implementation to experiment with.

-- 
--Guido van Rossum (python.org/~guido)

From tjreedy at udel.edu  Sat Jun 25 21:09:24 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 25 Jun 2011 15:09:24 -0400
Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy
In-Reply-To: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
References: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
Message-ID: <iu5bp5$paa$1@dough.gmane.org>

On 6/25/2011 12:33 PM, Vlad Riscutia wrote:
> I recently started looking at some ctypes issues. I dug a bit into
> http://bugs.python.org/issue6069 and then I found
> http://bugs.python.org/issue11920. They both boil down to the fact that
> bitfield allocation is up to the compiler, which is different in GCC and
> MSVC. Currently we have hard-coded allocation strategy based on paltform
> in cfields.c:
>
>>  if (bitsize /* this is a bitfield request */
>
>>         &&  *pfield_size /* we have a bitfield open */
>>  #ifdef MS_WIN32
>>         /* MSVC, GCC with -mms-bitfields */
>>         &&  dict->size * 8 == *pfield_size
>>  #else
>>         /* GCC */
>>         &&  dict->size * 8<= *pfield_size
>>  #endif
>>         &&  (*pbitofs + bitsize)<= *pfield_size) {
>>         /* continue bit field */
>>         fieldtype = CONT_BITFIELD;
>>  #ifndef MS_WIN32
>>     } else if (bitsize /* this is a bitfield request */
>>         &&  *pfield_size /* we have a bitfield open */
>>         &&  dict->size * 8>= *pfield_size
>>         &&  (*pbitofs + bitsize)<= dict->size * 8) {
>>         /* expand bit field */
>>         fieldtype = EXPAND_BITFIELD;
>>  #endif
>
> So when creating a bitfield structure, it's size can be different on
> Linux vs Windows.
>
> class MyStructure(ctypes.BigEndianStructure):
>      _pack_      = 1    # aligned to 8 bits, not ctypes default of 32
>      _fields_    = [
>                     ("Data0",   ctypes.c_uint32, 32),
>                     ("Data1",   ctypes.c_uint8, 3),
>                     ("Data2",   ctypes.c_uint16, 12),
>                    ]
>
> sizeof for above structure is 6 on GCC build and 7 on MSVC build. This
> leads to some confusion and issues, because we can't always interop
> correctly with code compiled with different compiler than the one Python
> is compiled with on the platform.

Just curious, are you saying that this is the 'cause' of the two bug 
reports, or 'just' something you discovered while investigating them?

 > Short term solution is to add a warning in the documentation about this.

For 2.7/3.2, yes.

 > Longer term though, I think it
> would be better to add a property on the Structure class for
> configurable allocation strategy, for example Native (default), GCC,
> MSVC and when allocating the bitfield, use given strategy. Native would
> behave similar to what happens now, but we would also allow GCC-style
> allocation on Windows for example and the ability to extend this if we
> ever run into similar issues with other compilers. This shouldn't be too
> difficult to implement, will be backwards compatible and it would
> improve interop. I would like to hear some opinions on this.

If this would allow the MSVC-compilied Python to better access dlls 
compiled with gcc (cygwin) on Windows, definitely -- in 3.3.
If the new feature is (currently) only useful on Windows, doc should say so.

-- 
Terry Jan Reedy


From riscutiavlad at gmail.com  Sat Jun 25 21:15:12 2011
From: riscutiavlad at gmail.com (Vlad Riscutia)
Date: Sat, 25 Jun 2011 12:15:12 -0700
Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy
In-Reply-To: <iu5bp5$paa$1@dough.gmane.org>
References: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
	<iu5bp5$paa$1@dough.gmane.org>
Message-ID: <BANLkTikjhfPeQCiMxVHC9-k8zRx7QKrLiA@mail.gmail.com>

This is the cause of both bug reports and yes, it should improve interop
with GCC-compiled code on Windows. My point is that this is not a platform
thing, it's more of a compiler thing. Currently issues appear on Windows for
interop between MSVC Python and other GCC code but since bitfield allocation
is up to the compiler, it could appear on any other platform due to
different compilers being used.

On Sat, Jun 25, 2011 at 12:09 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 6/25/2011 12:33 PM, Vlad Riscutia wrote:
>
>> I recently started looking at some ctypes issues. I dug a bit into
>> http://bugs.python.org/**issue6069 <http://bugs.python.org/issue6069> and
>> then I found
>> http://bugs.python.org/**issue11920 <http://bugs.python.org/issue11920>.
>> They both boil down to the fact that
>> bitfield allocation is up to the compiler, which is different in GCC and
>> MSVC. Currently we have hard-coded allocation strategy based on paltform
>> in cfields.c:
>>
>>   if (bitsize /* this is a bitfield request */
>>>
>>
>>         &&  *pfield_size /* we have a bitfield open */
>>>  #ifdef MS_WIN32
>>>        /* MSVC, GCC with -mms-bitfields */
>>>        &&  dict->size * 8 == *pfield_size
>>>  #else
>>>        /* GCC */
>>>        &&  dict->size * 8<= *pfield_size
>>>  #endif
>>>        &&  (*pbitofs + bitsize)<= *pfield_size) {
>>>        /* continue bit field */
>>>        fieldtype = CONT_BITFIELD;
>>>  #ifndef MS_WIN32
>>>    } else if (bitsize /* this is a bitfield request */
>>>        &&  *pfield_size /* we have a bitfield open */
>>>        &&  dict->size * 8>= *pfield_size
>>>        &&  (*pbitofs + bitsize)<= dict->size * 8) {
>>>        /* expand bit field */
>>>        fieldtype = EXPAND_BITFIELD;
>>>  #endif
>>>
>>
>> So when creating a bitfield structure, it's size can be different on
>> Linux vs Windows.
>>
>> class MyStructure(ctypes.**BigEndianStructure):
>>     _pack_      = 1    # aligned to 8 bits, not ctypes default of 32
>>     _fields_    = [
>>                    ("Data0",   ctypes.c_uint32, 32),
>>                    ("Data1",   ctypes.c_uint8, 3),
>>                    ("Data2",   ctypes.c_uint16, 12),
>>                   ]
>>
>> sizeof for above structure is 6 on GCC build and 7 on MSVC build. This
>> leads to some confusion and issues, because we can't always interop
>> correctly with code compiled with different compiler than the one Python
>> is compiled with on the platform.
>>
>
> Just curious, are you saying that this is the 'cause' of the two bug
> reports, or 'just' something you discovered while investigating them?
>
>
> > Short term solution is to add a warning in the documentation about this.
>
> For 2.7/3.2, yes.
>
>
> > Longer term though, I think it
>
>> would be better to add a property on the Structure class for
>> configurable allocation strategy, for example Native (default), GCC,
>> MSVC and when allocating the bitfield, use given strategy. Native would
>> behave similar to what happens now, but we would also allow GCC-style
>> allocation on Windows for example and the ability to extend this if we
>> ever run into similar issues with other compilers. This shouldn't be too
>> difficult to implement, will be backwards compatible and it would
>> improve interop. I would like to hear some opinions on this.
>>
>
> If this would allow the MSVC-compilied Python to better access dlls
> compiled with gcc (cygwin) on Windows, definitely -- in 3.3.
> If the new feature is (currently) only useful on Windows, doc should say
> so.
>
> --
> Terry Jan Reedy
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> riscutiavlad%40gmail.com<http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110625/069cf9d8/attachment.html>

From greg.ewing at canterbury.ac.nz  Sun Jun 26 01:33:13 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 26 Jun 2011 11:33:13 +1200
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <20110625153157.4B1063A4093@sparrow.telecommunity.com>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>
	<20110625153157.4B1063A4093@sparrow.telecommunity.com>
Message-ID: <4E067039.3060101@canterbury.ac.nz>

P.J. Eby wrote:
> At 10:46 AM 6/25/2011 +1000, Nick Coghlan wrote:
> 
>> Indeed, PEP 380 is *really* hard to do properly without language
>> support.
> 
> No, it isn't.  You add a decorator, a 'from_' class, and a 'return_' 
> function, and there you go.  (See my previous code sketches here in 
> early PEP 380 discussions.)

Will it handle *all* of the generator protocol correctly,
including send(), exception handling, and generator closing?

Also, how efficient would it be? A major benefit of a built-in
implementation is that it can be almost as fast as using the
sub-generator directly.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sun Jun 26 01:37:05 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 26 Jun 2011 11:37:05 +1200
Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy
In-Reply-To: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
References: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
Message-ID: <4E067121.9070802@canterbury.ac.nz>

Vlad Riscutia wrote:
> Longer term though, I think it 
> would be better to add a property on the Structure class for 
> configurable allocation strategy, for example Native (default), GCC, 
> MSVC

It could also be good to have a mode which lets you specify
*exactly* how the bits are laid out, independent of any
particular compiler.

-- 
Greg

From ncoghlan at gmail.com  Sun Jun 26 05:16:35 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 26 Jun 2011 13:16:35 +1000
Subject: [Python-Dev] PEP 380 acceptance (was Re: EuroPython Language Summit
	report)
Message-ID: <BANLkTin1Voeg-7hBUOSwrMQNwu9BqE-nQA@mail.gmail.com>

On Sun, Jun 26, 2011 at 2:44 AM, Guido van Rossum <guido at python.org> wrote:
> Let me cut this short. PEP 380 is pretty much approved. I know there
> are a few details worth quibbling over, but they are not going to
> jeopardize acceptance of the PEP. We are waiting for an implementation
> in Python 3.3. In fact, I wouldn't mind at this point if someone took
> their best effort at an implementation and checked it into the 3.3
> branch, and we can do the quibbling over the details while we have a
> working implementation to experiment with.

Based on this message, I've bumped PEP 380 to Accepted, and I'm now
working on committing Renaud Blanch's forward port of Greg's original
patch (see http://bugs.python.org/issue11682).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Sun Jun 26 06:29:57 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 26 Jun 2011 14:29:57 +1000
Subject: [Python-Dev] PEP 380 acceptance (was Re: EuroPython Language
 Summit report)
In-Reply-To: <BANLkTin1Voeg-7hBUOSwrMQNwu9BqE-nQA@mail.gmail.com>
References: <BANLkTin1Voeg-7hBUOSwrMQNwu9BqE-nQA@mail.gmail.com>
Message-ID: <BANLkTikzx4w3GS9FfHM+q4GQ1Ur-bU5L_A@mail.gmail.com>

On Sun, Jun 26, 2011 at 1:16 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Jun 26, 2011 at 2:44 AM, Guido van Rossum <guido at python.org> wrote:
>> Let me cut this short. PEP 380 is pretty much approved. I know there
>> are a few details worth quibbling over, but they are not going to
>> jeopardize acceptance of the PEP. We are waiting for an implementation
>> in Python 3.3. In fact, I wouldn't mind at this point if someone took
>> their best effort at an implementation and checked it into the 3.3
>> branch, and we can do the quibbling over the details while we have a
>> working implementation to experiment with.
>
> Based on this message, I've bumped PEP 380 to Accepted, and I'm now
> working on committing Renaud Blanch's forward port of Greg's original
> patch (see http://bugs.python.org/issue11682).

I hit a snag with this. The real tests of the PEP 380 functionality
aren't currently part of the patch - they're a big set of "golden
output" tests in the zipfile hosted on Greg's site. Those need to be
refactored into proper unittest or doctest based additions to the test
suite and incorporated into the patch before I could commit this with
a clear conscience. That's not going to be as quick as I first
thought.

Renaud's patch mostly applies cleanly at the moment - the only change
is that the "#endif" for the Py_LIMITED_API check needs to be moved in
pyerrors.h so it also covers the new StopIteration struct definition.

Regards,
Nick.

[1] http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/yield_from.html

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From barry at python.org  Sun Jun 26 13:30:31 2011
From: barry at python.org (Barry Warsaw)
Date: Sun, 26 Jun 2011 12:30:31 +0100
Subject: [Python-Dev] [Python-checkins] peps: Record Guido's acceptance
	of PEP 380
In-Reply-To: <E1QafcC-0001lU-1o@dinsdale.python.org>
References: <E1QafcC-0001lU-1o@dinsdale.python.org>
Message-ID: <20110626123031.5091420a@snowdog>

On Jun 26, 2011, at 05:02 AM, nick.coghlan wrote:

>http://hg.python.org/peps/rev/9f7a0b4e38a7
>changeset:   3889:9f7a0b4e38a7
>user:        Nick Coghlan <ncoghlan at gmail.com>
>date:        Sun Jun 26 13:02:17 2011 +1000
>summary:
>  Record Guido's acceptance of PEP 380
>
>files:
>  pep-0380.txt |  10 ++++++++--
>  1 files changed, 8 insertions(+), 2 deletions(-)
>
>
>diff --git a/pep-0380.txt b/pep-0380.txt
>--- a/pep-0380.txt
>+++ b/pep-0380.txt
>@@ -3,11 +3,11 @@
> Version: $Revision$
> Last-Modified: $Date$
> Author: Gregory Ewing <greg.ewing at canterbury.ac.nz>
>-Status: Draft
>+Status: Accepted
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 13-Feb-2009
>-Python-Version: 3.x
>+Python-Version: 3.3
> Post-History:

Please add a Resolution header as per PEP 1.

Thanks!
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110626/3a491781/attachment.pgp>

From ncoghlan at gmail.com  Sun Jun 26 15:55:02 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 26 Jun 2011 23:55:02 +1000
Subject: [Python-Dev] [Python-checkins] cpython (3.2): #11669: rephrase
 footnote in the Compound Statements page.
In-Reply-To: <E1QakrW-0004z8-RQ@dinsdale.python.org>
References: <E1QakrW-0004z8-RQ@dinsdale.python.org>
Message-ID: <BANLkTimZnXCy44JwxttcoOQWpSN-Df2gWg@mail.gmail.com>

On Sun, Jun 26, 2011 at 6:38 PM, ezio.melotti
<python-checkins at python.org> wrote:
> -.. [#] The exception is propagated to the invocation stack only if there is no
> - ? :keyword:`finally` clause that negates the exception.
> +.. [#] The exception is propagated to the invocation stack unless
> + ? there is a :keyword:`finally` clause which happens to raise another
> + ? exception. That new exception causes the old one to be lost.

I believe the footnote was talking about this case:

>>> def f():
...   try:
...     raise Exception()
...   finally:
...     return "What exception?"
...
>>> f()
'What exception?'

The new wording doesn't accurately reflect that.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From sandro.tosi at gmail.com  Sun Jun 26 16:02:52 2011
From: sandro.tosi at gmail.com (Sandro Tosi)
Date: Sun, 26 Jun 2011 16:02:52 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.2): #11669: rephrase
 footnote in the Compound Statements page.
In-Reply-To: <BANLkTimZnXCy44JwxttcoOQWpSN-Df2gWg@mail.gmail.com>
References: <E1QakrW-0004z8-RQ@dinsdale.python.org>
	<BANLkTimZnXCy44JwxttcoOQWpSN-Df2gWg@mail.gmail.com>
Message-ID: <BANLkTi=s2D08cBGSJCqiv8LAknYY_Jc4fg@mail.gmail.com>

Hi Nick,
given I'm "guilty" for this patch, I'd reply :)

On Sun, Jun 26, 2011 at 15:55, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Jun 26, 2011 at 6:38 PM, ezio.melotti
> <python-checkins at python.org> wrote:
>> -.. [#] The exception is propagated to the invocation stack only if there is no
>> - ? :keyword:`finally` clause that negates the exception.
>> +.. [#] The exception is propagated to the invocation stack unless
>> + ? there is a :keyword:`finally` clause which happens to raise another
>> + ? exception. That new exception causes the old one to be lost.
>
> I believe the footnote was talking about this case:
>
>>>> def f():
> ... ? try:
> ... ? ? raise Exception()
> ... ? finally:
> ... ? ? return "What exception?"
> ...
>>>> f()
> 'What exception?'
>
> The new wording doesn't accurately reflect that.

I gave my interpretation of the footnote at:
http://bugs.python.org/issue11669#msg139092 . Does this clarify it?

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

From senthil at uthcode.com  Sun Jun 26 20:52:42 2011
From: senthil at uthcode.com (Senthil Kumaran)
Date: Sun, 26 Jun 2011 11:52:42 -0700
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members in
	documentation
Message-ID: <20110626185242.GB2710@mathmagic>

Hello! 

http://bugs.python.org/issue10403 is a documentation bug which talks
about using the term 'attribute' instead of the term 'member' when it
denotes the class attributes. Agreed.

But the discussion goes on to mention that,

"Members and methods" should just be "attributes".

I find this bit odd. If the two terms are used together, then
replacing it with attributes is fine. But the term 'methods' cannot be
replaced with 'attributes' as it changes the meaning.

Take this case,

   :class:`BZ2File` provides all of the methods specified by the
   :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`.
   Iteration and the :keyword:`with` statement are supported.

is correct, whereas replacing "methods with attributes" would make it as

   :class:`BZ2File` provides all of the attributes specified by the
   :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`.
   Iteration and the :keyword:`with` statement are supported.

It does not seem correct.

My stance is, "It is attributes instead of term members and the term method
when it denotes methods can be left as such." Can I still hold on to that and
modify the patch which Adam has submitted or the 'attribute' substitution
everywhere makes sense?


Thanks,
Senthil

From tjreedy at udel.edu  Sun Jun 26 23:52:36 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 26 Jun 2011 17:52:36 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <20110626185242.GB2710@mathmagic>
References: <20110626185242.GB2710@mathmagic>
Message-ID: <iu89n5$eg3$1@dough.gmane.org>

On 6/26/2011 2:52 PM, Senthil Kumaran wrote:

> http://bugs.python.org/issue10403 is a documentation bug which talks
> about using the term 'attribute' instead of the term 'member' when it
> denotes the class attributes. Agreed.
>
> But the discussion goes on to mention that,
>
> "Members and methods" should just be "attributes".

The terms 'member' ('data attribute' in modern terms) and 'method' go 
back to original Python when builtin types (or instances thereof) had 
members, methods, neither, or possibly both (but I do not remember 
anything with both). I believe there were separate builtin functions for 
retrieving them. "Member' is obsolete; 'method' definitely is not.

> I find this bit odd. If the two terms are used together, then
> replacing it with attributes is fine.

Agreed.

> But the term 'methods' cannot be
> replaced with 'attributes' as it changes the meaning.

Also agreed.

> Take this case,
>
>     :class:`BZ2File` provides all of the methods specified by the
>     :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`.
>     Iteration and the :keyword:`with` statement are supported.
>
> is correct, whereas replacing "methods with attributes" would make it as
>
>     :class:`BZ2File` provides all of the attributes specified by the
>     :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`.
>     Iteration and the :keyword:`with` statement are supported.
>
> It does not seem correct.

It may not even be correct. I would leave it as is unless there are 
inherited data attributes so that the correction makes more sense than 
the original. A blind change of 'method' to 'attribute' is wrong.

> My stance is, "It is attributes instead of term members and the term method
> when it denotes methods can be left as such." Can I still hold on to that and
> modify the patch which Adam has submitted

Yes.

> or the 'attribute' substitution everywhere makes sense?

No.

My strong history-based opinions ;-).

-- 
Terry Jan Reedy


From riscutiavlad at gmail.com  Mon Jun 27 02:48:23 2011
From: riscutiavlad at gmail.com (Vlad Riscutia)
Date: Sun, 26 Jun 2011 17:48:23 -0700
Subject: [Python-Dev] ctypes: Configurable bitfield allocation strategy
In-Reply-To: <4E067121.9070802@canterbury.ac.nz>
References: <BANLkTinN2kr2_qF1Lcg5G8ZsW+0Mc1VH-g@mail.gmail.com>
	<4E067121.9070802@canterbury.ac.nz>
Message-ID: <BANLkTi=QEvc8sMyEaLJfagYMPoAY0Nvftw@mail.gmail.com>

Well it's not really layout, because alignment is handled by pack option. It
is how the field gets allocated. At this point I believe it will be more
complex to come up with custom allocation option, precisely because it's up
to each compiler to allocate the structure. Such flexibility will add a lot
of complexity and it might not payoff. This is debatable, but I would go
with a 3 option strategy at this point - native, GCC, MSVC and if we
actually find interop issues with other compilers we can look into custom
allocation.

I will try to refactor the C code to more easily accommodate a mode option
(while leaving behavior the same for now), then we can decide how the
library interface should look like.

Thank you,
Vlad

On Sat, Jun 25, 2011 at 4:37 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Vlad Riscutia wrote:
>
>> Longer term though, I think it would be better to add a property on the
>> Structure class for configurable allocation strategy, for example Native
>> (default), GCC, MSVC
>>
>
> It could also be good to have a mode which lets you specify
> *exactly* how the bits are laid out, independent of any
> particular compiler.
>
> --
> Greg
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> riscutiavlad%40gmail.com<http://mail.python.org/mailman/options/python-dev/riscutiavlad%40gmail.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110626/e3983c5e/attachment.html>

From ncoghlan at gmail.com  Mon Jun 27 03:27:36 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Jun 2011 11:27:36 +1000
Subject: [Python-Dev] [Python-checkins] cpython (3.2): #11669: rephrase
 footnote in the Compound Statements page.
In-Reply-To: <BANLkTi=s2D08cBGSJCqiv8LAknYY_Jc4fg@mail.gmail.com>
References: <E1QakrW-0004z8-RQ@dinsdale.python.org>
	<BANLkTimZnXCy44JwxttcoOQWpSN-Df2gWg@mail.gmail.com>
	<BANLkTi=s2D08cBGSJCqiv8LAknYY_Jc4fg@mail.gmail.com>
Message-ID: <BANLkTinE9hq_05LcaNPgK90bLdDVzXqTKQ@mail.gmail.com>

On Mon, Jun 27, 2011 at 12:02 AM, Sandro Tosi <sandro.tosi at gmail.com> wrote:
> I gave my interpretation of the footnote at:
> http://bugs.python.org/issue11669#msg139092 . Does this clarify it?

No, because while there *are* ways a finally clause can kill an
exception completely, reraising another exception is not really one of
them (as we set __context__ appropriately in that case, even if it
means forcing instantiation of an as yet unrealised exception).

>From PEP 3134:

"This PEP handles exceptions that occur during 'except' blocks and
'finally' blocks in the same way.  Reading the traceback makes it
clear where the exceptions occurred, so additional mechanisms for
distinguishing the two cases would only add unnecessary complexity."

And from the interactive prompt:

>>> try:
...   raise TypeError
... except TypeError:
...   raise ValueError
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
TypeError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
ValueError
>>> try:
...   raise TypeError
... finally:
...   raise ValueError
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
TypeError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
ValueError

Note that PEP 3134 exception chaining means the original exception is
not lost in either case - it is available via the __context__
attribute of the subsequent exception.

However, *other* control flow statements in a finally clause *can*
mean that an exception is lost completely, just as if it had been
caught and not reraised:

- the "return" example from my first message ('yield' will allow the
exception to propagate when the generator is resumed)
- 'break' in a loop ('continue' is not permitted inside a 'finally' clause)

The old wording was at least vaguely right ("there are ways that a
finally clause can kill an exception, but we aren't going to tell you
what they are"). The new wording is precisely wrong, as it excludes
the return and break cases (in both 2.x and 3.x) and, in 3.x, fails to
note that the old exception isn't lost completely due to exception
chaining (even if it is indeed hidden from any exception handlers in
the current call stack).

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Jun 27 03:32:32 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Jun 2011 11:32:32 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <iu89n5$eg3$1@dough.gmane.org>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>
Message-ID: <BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>

On Mon, Jun 27, 2011 at 7:52 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> or the 'attribute' substitution everywhere makes sense?
>
> No.
>
> My strong history-based opinions ;-).

+1 to what Terry said.

"Members" is a historical relic that is best replaced by "attributes"
or "data attributes" if we want to explicitly exclude methods for some
reason. "Methods" is a subset of attributes that explicitly excludes
data attributes.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From pje at telecommunity.com  Mon Jun 27 03:43:49 2011
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 26 Jun 2011 21:43:49 -0400
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <20110625163242.349EC2506ED@webabinitio.net>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<BANLkTi=M+4j7=9Kvt2YKBNO=PmCVeZOzxA@mail.gmail.com>
	<20110625153157.4B1063A4093@sparrow.telecommunity.com>
	<20110625163242.349EC2506ED@webabinitio.net>
Message-ID: <20110627014358.375273A403A@sparrow.telecommunity.com>

At 12:32 PM 6/25/2011 -0400, R. David Murray wrote:
>So your proposed code would allow me, when writing a generator in
>my code, do something that would allow me to yield up all the
>values from an arbitrary generator I'm calling, over which I have
>no control (ie: I can't modify its code)?

With a decorator on your own function, yes.  See:

   http://mail.python.org/pipermail/python-dev/2010-July/102320.html

for details.  Mostly, though, that proposal was a suggestion for how 
the "optimized" implementation would work - i.e., a suggestion that 
PEP 380 be implemented that way under the hood, by implicitly turning 
'yield from' into 'yield From()' and wrapping the generator itself 
with another From() instance.

(IOW, that was a proposal for how to avoid the extra overhead of 
recursive yielding in a series of nested yield-from's.)


From solipsis at pitrou.net  Mon Jun 27 10:24:28 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 27 Jun 2011 10:24:28 +0200
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
Message-ID: <20110627102428.6fc1955b@msiwind>

Le Mon, 27 Jun 2011 11:32:32 +1000,
Nick Coghlan <ncoghlan at gmail.com> a ?crit :

> On Mon, Jun 27, 2011 at 7:52 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> >> or the 'attribute' substitution everywhere makes sense?
> >
> > No.
> >
> > My strong history-based opinions ;-).
> 
> +1 to what Terry said.
> 
> "Members" is a historical relic that is best replaced by "attributes"
> or "data attributes" if we want to explicitly exclude methods for some
> reason. "Methods" is a subset of attributes that explicitly excludes
> data attributes.

While I know it is technically right, I find it a bit strange to refer to
methods as "attributes". We're describing an API, not the inner working of
the object model. Also, people just discovering Python will probably be a
bit surprised if we start refer to methods as "attributes".

FWIW, I tend to understand "members" as "methods + attributes", which makes
it a nice term to use for that purpose.

Regards

Antoine.



From p.f.moore at gmail.com  Mon Jun 27 10:47:05 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 27 Jun 2011 09:47:05 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <20110627102428.6fc1955b@msiwind>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
Message-ID: <BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>

On 27 June 2011 09:24, Antoine Pitrou <solipsis at pitrou.net> wrote:
> While I know it is technically right, I find it a bit strange to refer to
> methods as "attributes". We're describing an API, not the inner working of
> the object model. Also, people just discovering Python will probably be a
> bit surprised if we start refer to methods as "attributes".

+1

> FWIW, I tend to understand "members" as "methods + attributes", which makes
> it a nice term to use for that purpose.

+1

Paul.

From fuzzyman at voidspace.org.uk  Mon Jun 27 11:42:54 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 27 Jun 2011 10:42:54 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <20110627102428.6fc1955b@msiwind>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
Message-ID: <4E08509E.1030401@voidspace.org.uk>

On 27/06/2011 09:24, Antoine Pitrou wrote:
> Le Mon, 27 Jun 2011 11:32:32 +1000,
> Nick Coghlan<ncoghlan at gmail.com>  a ?crit :
>
>> On Mon, Jun 27, 2011 at 7:52 AM, Terry Reedy<tjreedy at udel.edu>  wrote:
>>>> or the 'attribute' substitution everywhere makes sense?
>>> No.
>>>
>>> My strong history-based opinions ;-).
>> +1 to what Terry said.
>>
>> "Members" is a historical relic that is best replaced by "attributes"
>> or "data attributes" if we want to explicitly exclude methods for some
>> reason. "Methods" is a subset of attributes that explicitly excludes
>> data attributes.
> While I know it is technically right, I find it a bit strange to refer to
> methods as "attributes". We're describing an API, not the inner working of
> the object model. Also, people just discovering Python will probably be a
> bit surprised if we start refer to methods as "attributes".
>
> FWIW, I tend to understand "members" as "methods + attributes", which makes
> it a nice term to use for that purpose.

That is my understanding / use of the terms as well.

Michael

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From phd at phdru.name  Mon Jun 27 11:45:26 2011
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 27 Jun 2011 13:45:26 +0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <20110627102428.6fc1955b@msiwind>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
Message-ID: <20110627094526.GA7950@iskra.aviel.ru>

On Mon, Jun 27, 2011 at 10:24:28AM +0200, Antoine Pitrou wrote:
> FWIW, I tend to understand "members" as "methods + attributes", which makes
> it a nice term to use for that purpose.

   That's my feeling too.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From tom.whittock at gmail.com  Mon Jun 27 12:48:35 2011
From: tom.whittock at gmail.com (Tom Whittock)
Date: Mon, 27 Jun 2011 11:48:35 +0100
Subject: [Python-Dev] Patching builtin_id to allow for C proxy objects?
Message-ID: <BANLkTi=CzT0L3cufH7kvjwSbGs-QTKjgBQ@mail.gmail.com>

Hi all.

I'm writing a module to proxy C++ objects into Python for a large C++
application. There are hundreds of thousands of C++ objects, some of
which are temporary while others are very long lived.

Currently every time one of these objects is accessed from Python, a
new "myproxy" instance is created. So if I were to access the same
field of an object twice, I would receive two python objects proxying
the same underlying C++ object. This messes up "id" and "is", and is
causing me issues when, for example, I run into circular references
when enoding json or otherwise attempt to determine whether two proxy
objects refer to the same C++ object.

I can't see how to cache the "myproxy" objects instead of returning
new instances - due to the architecture of the C++ application,
there's no weak reference support at all, and the number of objects is
very large.

My current plan would be for me to override the id builtin to return
the underlying C++ object instance pointer instead of the PyObject
instance pointer in the case of the "myproxy" object type, probably
using a new type method slot tp_id or similar. The old behaviour would
be unchanged for all other types, naturally. I'd also need to alter
ceval.c to use builtin_id instead of the raw pointer for comparison
when using PyCmp_IS and PyCmp_IS_NOT. I can see that there could very
well be many other sites throughout the C source where the pointer was
directly compared, and would cause interesting issues for me down the
line. I'm just not sure what else to try.

I'd like to know if I'm being laughably naive or not before I went
about this plan, and whether it'd be worthwhile contributing the patch
back, considering the number of potentially overridden-id-unaware
areas throught the rest of the python source base.

Thanks.
Tom.

From stefan_ml at behnel.de  Mon Jun 27 13:27:30 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 27 Jun 2011 13:27:30 +0200
Subject: [Python-Dev] Patching builtin_id to allow for C proxy objects?
In-Reply-To: <BANLkTi=CzT0L3cufH7kvjwSbGs-QTKjgBQ@mail.gmail.com>
References: <BANLkTi=CzT0L3cufH7kvjwSbGs-QTKjgBQ@mail.gmail.com>
Message-ID: <iu9pf2$i9j$1@dough.gmane.org>

Tom Whittock, 27.06.2011 12:48:
> I'm writing a module to proxy C++ objects into Python for a large C++
> application. There are hundreds of thousands of C++ objects, some of
> which are temporary while others are very long lived.
>
> Currently every time one of these objects is accessed from Python, a
> new "myproxy" instance is created. So if I were to access the same
> field of an object twice, I would receive two python objects proxying
> the same underlying C++ object. This messes up "id" and "is"

Note that "is" actually compares the addresses, not the id().


> and is
> causing me issues when, for example, I run into circular references
> when enoding json or otherwise attempt to determine whether two proxy
> objects refer to the same C++ object.
>
> I can't see how to cache the "myproxy" objects instead of returning
> new instances - due to the architecture of the C++ application,
> there's no weak reference support at all, and the number of objects is
> very large.
>
> My current plan would be for me to override the id builtin to return
> the underlying C++ object instance pointer instead of the PyObject
> instance pointer in the case of the "myproxy" object type

Where would you get the proxy object from anyway?

IMHO, there are two obvious way get what you want: map the C++ object 
address (integer!) to a proxy object using a dict, or use a backpointer 
from the C++ object to its proxy. The second is substantially faster, but 
may require changes to the C++ class struct.

I don't see how changes to CPython's core can help you here.

Stefan


From greg.ewing at canterbury.ac.nz  Mon Jun 27 14:02:33 2011
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Jun 2011 00:02:33 +1200
Subject: [Python-Dev] Patching builtin_id to allow for C proxy objects?
In-Reply-To: <BANLkTi=CzT0L3cufH7kvjwSbGs-QTKjgBQ@mail.gmail.com>
References: <BANLkTi=CzT0L3cufH7kvjwSbGs-QTKjgBQ@mail.gmail.com>
Message-ID: <4E087159.7000009@canterbury.ac.nz>

Tom Whittock wrote:

> Currently every time one of these objects is accessed from Python, a
> new "myproxy" instance is created. So if I were to access the same
> field of an object twice, I would receive two python objects proxying
> the same underlying C++ object.

Perhaps you could use a WeakValueDictionary to keep a mapping
from a C++ object address to its Python proxy. Then as long as
a proxy object is alive, accessing the same C++ object again
will get you the same proxy object. When there are no longer
any references to the proxy object from Python, it will go
away. The next time you access that C++ object you'll get a
new proxy, but that won't matter, because the original proxy
is no longer around to compare it with.

This depends on there being some way for the proxy object
to ensure that the C++ object remains alive as long as it
does.

It also won't solve the problem of keeping the id of the
proxy for longer than the proxy exists, but that's probably
not something you should be relying on anyway. The id of
*any* Python object is only valid while the object lives,
and if it's still alive you have a real reference somewhere
that you can use instead of the id for identity testing.

-- 
Greg

From tom.whittock at gmail.com  Mon Jun 27 14:31:06 2011
From: tom.whittock at gmail.com (Tom Whittock)
Date: Mon, 27 Jun 2011 13:31:06 +0100
Subject: [Python-Dev] Patching builtin_id to allow for C proxy objects?
In-Reply-To: <4E087159.7000009@canterbury.ac.nz>
References: <BANLkTi=CzT0L3cufH7kvjwSbGs-QTKjgBQ@mail.gmail.com>
	<4E087159.7000009@canterbury.ac.nz>
Message-ID: <BANLkTinNUUsKMMHmb0uP4qCLS9h01a0U4w@mail.gmail.com>

Hi Greg thanks for your quick reply.

> Perhaps you could use a WeakValueDictionary to keep a mapping
> from a C++ object address to its Python proxy.

Thank you, I'll implement this and see whether it works out. I'll
certainly be better off if it does. I was avoiding holding weak
references due to perhaps unfounded concerns about increasing the
lifetime and speed/memory impact of certain temporary objects which
are created at very high frequency. I'll test it and see before diving
into messing with id. But now I'm thinking about it again, I can see a
plan for not needing to affect that pathway at all.

Seems I fell into the trap of making things too complicated for myself.

> It also won't solve the problem of keeping the id of the
> proxy for longer than the proxy exists, but that's probably
> not something you should be relying on anyway. The id of
> *any* Python object is only valid while the object lives,
> and if it's still alive you have a real reference somewhere
> that you can use instead of the id for identity testing.

Thanks, yes. I'm actually kind of concerned about the usage of id in
the markers set which the json library uses for circular referencing
checks for exactly this reason. It seems to assume that the objects
lifetime lasts for the duration of the encoding operation. I have no
idea if that's actually the case in my situation, where data members
are property getters producing probably very short lived proxies
generated from C++. I guess I'll find out :)

Thanks again,
Tom.

From tom.whittock at gmail.com  Mon Jun 27 16:05:37 2011
From: tom.whittock at gmail.com (Tom Whittock)
Date: Mon, 27 Jun 2011 15:05:37 +0100
Subject: [Python-Dev] Patching builtin_id to allow for C proxy objects?
In-Reply-To: <BANLkTinNUUsKMMHmb0uP4qCLS9h01a0U4w@mail.gmail.com>
References: <BANLkTi=CzT0L3cufH7kvjwSbGs-QTKjgBQ@mail.gmail.com>
	<4E087159.7000009@canterbury.ac.nz>
	<BANLkTinNUUsKMMHmb0uP4qCLS9h01a0U4w@mail.gmail.com>
Message-ID: <BANLkTik3J+xmqTu05VVkbdjLJ_Y4WpO2cA@mail.gmail.com>

Hi again.

Just to let you know that Greg's suggestion worked beautifully - I
guess my id idea was just me trying to make life hard for myself.

My concerns over the json modules usage of id seem unjustified, as
circular references are detected now that the weak reference
dictionary is in place.

Thanks for your help, and sorry for bothering dev with something which
was a regular python programming issue after all.

Tom.

On 27 June 2011 13:31, Tom Whittock <tom.whittock at gmail.com> wrote:
> Hi Greg thanks for your quick reply.
>
>> Perhaps you could use a WeakValueDictionary to keep a mapping
>> from a C++ object address to its Python proxy.
>
> Thank you, I'll implement this and see whether it works out. I'll
> certainly be better off if it does. I was avoiding holding weak
> references due to perhaps unfounded concerns about increasing the
> lifetime and speed/memory impact of certain temporary objects which
> are created at very high frequency. I'll test it and see before diving
> into messing with id. But now I'm thinking about it again, I can see a
> plan for not needing to affect that pathway at all.
>
> Seems I fell into the trap of making things too complicated for myself.
>
>> It also won't solve the problem of keeping the id of the
>> proxy for longer than the proxy exists, but that's probably
>> not something you should be relying on anyway. The id of
>> *any* Python object is only valid while the object lives,
>> and if it's still alive you have a real reference somewhere
>> that you can use instead of the id for identity testing.
>
> Thanks, yes. I'm actually kind of concerned about the usage of id in
> the markers set which the json library uses for circular referencing
> checks for exactly this reason. It seems to assume that the objects
> lifetime lasts for the duration of the encoding operation. I have no
> idea if that's actually the case in my situation, where data members
> are property getters producing probably very short lived proxies
> generated from C++. I guess I'll find out :)
>
> Thanks again,
> Tom.
>

From rdmurray at bitdance.com  Mon Jun 27 16:08:21 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 27 Jun 2011 10:08:21 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
Message-ID: <20110627140826.877032505A3@webabinitio.net>

On Mon, 27 Jun 2011 09:47:05 +0100, Paul Moore <p.f.moore at gmail.com> wrote:
> On 27 June 2011 09:24, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > While I know it is technically right, I find it a bit strange to refer to
> > methods as "attributes". We're describing an API, not the inner working of
> > the object model. Also, people just discovering Python will probably be a
> > bit surprised if we start refer to methods as "attributes".
> 
> +1
> 
> > FWIW, I tend to understand "members" as "methods + attributes", which makes
> > it a nice term to use for that purpose.
> 
> +1

Wow, all these people who like 'members', and I can't think of ever
using that term in a Python context.

While I agree that using 'attribute' when only methods are being discussed
would most likely be confusing, and that it can be tricky to clearly
word things when both are being discussed, the existence in the language
of getattr, setattr, and related methods argues against using the term
'members'.

'data attributes' can so easily become something else in Python...it
seems to me that the only real difference between 'data attributes' and
'method attributes' in Python is that the latter can be called and the
former can't.  But even that is not an accurate distinction, since a
'data attribute' could, in fact, return a callable.

I guess what I'm saying is that I am more comfortable calling them
all attributes than calling them all members.  The term 'members'
isn't used anywhere in the language itself, as far as I can recall,
whereas getattr and setattr are evidence that the language considers
them all attributes.  I think we do the documentation readers a
disservice by obscuring that fact by using other terminology.

--
R. David Murray           http://www.bitdance.com

From rob.cliffe at btinternet.com  Mon Jun 27 16:19:42 2011
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Mon, 27 Jun 2011 15:19:42 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <20110627140826.877032505A3@webabinitio.net>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
Message-ID: <4E08917E.3020509@btinternet.com>

On 27/06/2011 15:08, R. David Murray wrote:
> Wow, all these people who like 'members', and I can't think of ever
> using that term in a Python context.
>
> While I agree that using 'attribute' when only methods are being discussed
> would most likely be confusing, and that it can be tricky to clearly
> word things when both are being discussed, the existence in the language
> of getattr, setattr, and related methods argues against using the term
> 'members'.
>
> 'data attributes' can so easily become something else in Python...it
> seems to me that the only real difference between 'data attributes' and
> 'method attributes' in Python is that the latter can be called and the
> former can't.  But even that is not an accurate distinction, since a
> 'data attribute' could, in fact, return a callable.
>
> I guess what I'm saying is that I am more comfortable calling them
> all attributes than calling them all members.  The term 'members'
> isn't used anywhere in the language itself, as far as I can recall,
> whereas getattr and setattr are evidence that the language considers
> them all attributes.  I think we do the documentation readers a
> disservice by obscuring that fact by using other terminology.
>
+1.

'function attributes' ?  'def attributes' ?  Or just stick with 'method 
attributes' ?


From fuzzyman at voidspace.org.uk  Mon Jun 27 16:27:12 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 27 Jun 2011 15:27:12 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <20110627140826.877032505A3@webabinitio.net>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
Message-ID: <4E089340.7060004@voidspace.org.uk>

On 27/06/2011 15:08, R. David Murray wrote:
> On Mon, 27 Jun 2011 09:47:05 +0100, Paul Moore<p.f.moore at gmail.com>  wrote:
>> On 27 June 2011 09:24, Antoine Pitrou<solipsis at pitrou.net>  wrote:
>>> While I know it is technically right, I find it a bit strange to refer to
>>> methods as "attributes". We're describing an API, not the inner working of
>>> the object model. Also, people just discovering Python will probably be a
>>> bit surprised if we start refer to methods as "attributes".
>> +1
>>
>>> FWIW, I tend to understand "members" as "methods + attributes", which makes
>>> it a nice term to use for that purpose.
>> +1
> Wow, all these people who like 'members', and I can't think of ever
> using that term in a Python context.
>
> While I agree that using 'attribute' when only methods are being discussed
> would most likely be confusing, and that it can be tricky to clearly
> word things when both are being discussed, the existence in the language
> of getattr, setattr, and related methods argues against using the term
> 'members'.
>
> 'data attributes' can so easily become something else in Python...it
> seems to me that the only real difference between 'data attributes' and
> 'method attributes' in Python is that the latter can be called and the
> former can't.  But even that is not an accurate distinction, since a
> 'data attribute' could, in fact, return a callable.
>
> I guess what I'm saying is that I am more comfortable calling them
> all attributes than calling them all members.  The term 'members'
> isn't used anywhere in the language itself, as far as I can recall,
> whereas getattr and setattr are evidence that the language considers
> them all attributes.  I think we do the documentation readers a
> disservice by obscuring that fact by using other terminology.
>

Well perhaps, but where does the language draw the distinction between 
attributes and "data attributes" as you all them (a term entirely new to 
me)? Only in the descriptor protocol and that term isn't used there 
(data-descriptors and non data-descriptors is terminology used in the 
documentation there).

If you're saying that data attributes isn't clear either (I couldn't 
quite tell from your email) then how *do* we draw a distinction. We 
could talk about instance attributes, non-descriptor class attributes 
and descriptors, but that terminology requires a reasonably advanced 
understanding of the Python data model.

I don't think that "all members, made up of attributes plus methods" is 
hard to understand. That's a great benefit. The fact that you can 
technically treat methods as attributes too is a minor detail.

All the best,

Michael Foord

> --
> R. David Murray           http://www.bitdance.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From ncoghlan at gmail.com  Mon Jun 27 16:32:59 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2011 00:32:59 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <20110627140826.877032505A3@webabinitio.net>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
Message-ID: <BANLkTi=dDeH=yuxB_8zguU8JHG2jwfu4Qg@mail.gmail.com>

On Tue, Jun 28, 2011 at 12:08 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> While I agree that using 'attribute' when only methods are being discussed
> would most likely be confusing, and that it can be tricky to clearly
> word things when both are being discussed, the existence in the language
> of getattr, setattr, and related methods argues against using the term
> 'members'.

Yep, to me "attribute" just means "something that can be accessed
using attribute notation". What it actually *is* is completely up for
grabs at that point.

> 'data attributes' can so easily become something else in Python...it
> seems to me that the only real difference between 'data attributes' and
> 'method attributes' in Python is that the latter can be called and the
> former can't. ?But even that is not an accurate distinction, since a
> 'data attribute' could, in fact, return a callable.
>
> I guess what I'm saying is that I am more comfortable calling them
> all attributes than calling them all members. ?The term 'members'
> isn't used anywhere in the language itself, as far as I can recall,
> whereas getattr and setattr are evidence that the language considers
> them all attributes. ?I think we do the documentation readers a
> disservice by obscuring that fact by using other terminology.

It's worse than that - the specific meaning of "members" in the
context of Python's history specifically *excludes* methods.

The superset is "attributes" - as noted, the names of the builtins and
magic methods make that terminology quite explicit.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Mon Jun 27 16:36:20 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2011 00:36:20 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E089340.7060004@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E089340.7060004@voidspace.org.uk>
Message-ID: <BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>

On Tue, Jun 28, 2011 at 12:27 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Well perhaps, but where does the language draw the distinction between
> attributes and "data attributes" as you all them (a term entirely new to
> me)? Only in the descriptor protocol and that term isn't used there
> (data-descriptors and non data-descriptors is terminology used in the
> documentation there).
>
> If you're saying that data attributes isn't clear either (I couldn't quite
> tell from your email) then how *do* we draw a distinction. We could talk
> about instance attributes, non-descriptor class attributes and descriptors,
> but that terminology requires a reasonably advanced understanding of the
> Python data model.
>
> I don't think that "all members, made up of attributes plus methods" is hard
> to understand. That's a great benefit. The fact that you can technically
> treat methods as attributes too is a minor detail.

It has almost no precedent in the Python context and what precedent it
does have is wrong (since it excluded methods).

And no, the fact that methods can be treated as attributes is not a
minor detail. It is *fundamental* to Python's object model that
*methods are not a special case of attribute access*. All attributes
work the same way, it is just the way functions implement the
descriptor protocol that makes instance methods behave the way they
do.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From benjamin at python.org  Mon Jun 27 16:38:10 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 27 Jun 2011 09:38:10 -0500
Subject: [Python-Dev] [Python-checkins] peps: Happy Tau Day folks! :)
In-Reply-To: <E1QbCja-0001P0-Gj@dinsdale.python.org>
References: <E1QbCja-0001P0-Gj@dinsdale.python.org>
Message-ID: <BANLkTikW_xktfpf=QNra+DX3QJArq8fEhQ@mail.gmail.com>

2011/6/27 nick.coghlan <python-checkins at python.org>:
> http://hg.python.org/peps/rev/1e3d663c67ee
> changeset: ? 3892:1e3d663c67ee
> user: ? ? ? ?Nick Coghlan <ncoghlan at gmail.com>
> date: ? ? ? ?Tue Jun 28 00:23:57 2011 +1000
> summary:
> ?Happy Tau Day folks! :)
>
> files:
> ?pep-0628.html | ?149 ++++++++++++++++++++++++++++++++++++++

You're not going to give us the source?



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Mon Jun 27 16:45:08 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2011 00:45:08 +1000
Subject: [Python-Dev] [Python-checkins] peps: Happy Tau Day folks! :)
In-Reply-To: <BANLkTikW_xktfpf=QNra+DX3QJArq8fEhQ@mail.gmail.com>
References: <E1QbCja-0001P0-Gj@dinsdale.python.org>
	<BANLkTikW_xktfpf=QNra+DX3QJArq8fEhQ@mail.gmail.com>
Message-ID: <BANLkTi=aNTgbYbOA38SMo-RtGchMyDdcaw@mail.gmail.com>

On Tue, Jun 28, 2011 at 12:38 AM, Benjamin Peterson <benjamin at python.org> wrote:
> You're not going to give us the source?

Just a teensy error with hg add, tab completion and not checking the
diff before committing. Fixed now, though :)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From rndblnch at gmail.com  Mon Jun 27 17:09:29 2011
From: rndblnch at gmail.com (renaud)
Date: Mon, 27 Jun 2011 15:09:29 +0000 (UTC)
Subject: [Python-Dev] PEP 380 acceptance (was Re: EuroPython Language
	Summit report)
References: <BANLkTin1Voeg-7hBUOSwrMQNwu9BqE-nQA@mail.gmail.com>
	<BANLkTikzx4w3GS9FfHM+q4GQ1Ur-bU5L_A@mail.gmail.com>
Message-ID: <loom.20110627T170547-215@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> I hit a snag with this. The real tests of the PEP 380 functionality
> aren't currently part of the patch - they're a big set of "golden
> output" tests in the zipfile hosted on Greg's site. Those need to be
> refactored into proper unittest or doctest based additions to the test
> suite and incorporated into the patch before I could commit this with
> a clear conscience. 

let me know if i can help.

> Renaud's patch mostly applies cleanly at the moment - the only change
> is that the "#endif" for the Py_LIMITED_API check needs to be moved in
> pyerrors.h so it also covers the new StopIteration struct definition.

if this helps, i've updated the patch to fix this.
https://bitbucket.org/rndblnch/cpython-pep380/changeset/6014d1720625

renaud



From stephen at xemacs.org  Mon Jun 27 17:32:07 2011
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 28 Jun 2011 00:32:07 +0900
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E089340.7060004@voidspace.org.uk>
	<BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>
Message-ID: <87vcvr1ea0.fsf@uwakimon.sk.tsukuba.ac.jp>

Nick Coghlan writes:

 > And no, the fact that methods can be treated as attributes is not a
 > minor detail. It is *fundamental* to Python's object model that
 > *methods are not a special case of attribute access*.

That's ambiguous.  I assume you mean "just a case of attribute access,
and not special in any way"?


From janssen at parc.com  Mon Jun 27 17:51:48 2011
From: janssen at parc.com (Bill Janssen)
Date: Mon, 27 Jun 2011 08:51:48 PDT
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E089340.7060004@voidspace.org.uk>
	<BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>
Message-ID: <25745.1309189908@parc.com>

Nick Coghlan <ncoghlan at gmail.com> wrote:

> And no, the fact that methods can be treated as attributes is not a
> minor detail. It is *fundamental* to Python's object model that
> *methods are not a special case of attribute access*. All attributes
> work the same way, it is just the way functions implement the
> descriptor protocol that makes instance methods behave the way they
> do.

Well put, Nick.  This paragraph is a good thing to read a couple of
times.

Bill

From rdmurray at bitdance.com  Mon Jun 27 18:56:32 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 27 Jun 2011 12:56:32 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <4E089340.7060004@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E089340.7060004@voidspace.org.uk>
Message-ID: <20110627165633.0A8652505A3@webabinitio.net>

On Mon, 27 Jun 2011 15:27:12 +0100, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> On 27/06/2011 15:08, R. David Murray wrote:
> > 'data attributes' can so easily become something else in Python...it
> > seems to me that the only real difference between 'data attributes' and
> > 'method attributes' in Python is that the latter can be called and the
> > former can't.  But even that is not an accurate distinction, since a
> > 'data attribute' could, in fact, return a callable.
> >
> > I guess what I'm saying is that I am more comfortable calling them
> > all attributes than calling them all members.  The term 'members'
> > isn't used anywhere in the language itself, as far as I can recall,
> > whereas getattr and setattr are evidence that the language considers
> > them all attributes.  I think we do the documentation readers a
> > disservice by obscuring that fact by using other terminology.
> 
> Well perhaps, but where does the language draw the distinction between 
> attributes and "data attributes" as you all them (a term entirely new to 

It doesn't, that's the point.  You'll note I put "data attributes"
in quotes :)

> me)? Only in the descriptor protocol and that term isn't used there 
> (data-descriptors and non data-descriptors is terminology used in the 
> documentation there).
> 
> If you're saying that data attributes isn't clear either (I couldn't 
> quite tell from your email) then how *do* we draw a distinction. We 
> could talk about instance attributes, non-descriptor class attributes 
> and descriptors, but that terminology requires a reasonably advanced 
> understanding of the Python data model.

That's why I said it could be difficult to find good wording when
discussing both methods and "other things".  Most people have a pretty
clear idea of what methods are, but the non-method stuff in Python does
not have any simple description that is also accurate.  Maybe 'non-method
attribute' is as close as we can get?

> I don't think that "all members, made up of attributes plus methods" is 
> hard to understand. That's a great benefit. The fact that you can 
> technically treat methods as attributes too is a minor detail.

Well, I would find that very hard to understand, since methods are
attributes, and as Nick said that is *fundamental* to the language,
not a minor detail.

--
R. David Murray           http://www.bitdance.com

From janssen at parc.com  Mon Jun 27 19:01:13 2011
From: janssen at parc.com (Bill Janssen)
Date: Mon, 27 Jun 2011 10:01:13 PDT
Subject: [Python-Dev] Snow Leopard buildbot failing again...
Message-ID: <26766.1309194073@parc.com>

I see that parc-snowleopard-1 went down again.  I've done a software
update, rebooted, and installed the latest buildslave, 0.8.4.  I can
ping dinsdale.python.org successfully from the machine.  However, when I
start the buildslave, I get this:

$ buildslave start ~/buildarea/
/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/_sslverify.py:5: DeprecationWarning: the md5 module is deprecated; use hashlib instead
  import itertools, md5
Following twistd.log until startup finished..
/Library/Python/2.6/site-packages/buildbot_slave-0.8.4-py2.6.egg/buildslave/scripts/logwatcher.py:67: PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
  env=os.environ,
/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/persisted/sob.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead
  import os, md5, sys
/Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/python/filepath.py:12: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
2011-06-27 09:45:31-0700 [-] Log opened.
2011-06-27 09:45:31-0700 [-] twistd 8.2.0 (/usr/bin/python 2.6.1) starting up.
2011-06-27 09:45:31-0700 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2011-06-27 09:45:31-0700 [-] Starting BuildSlave -- version: 0.8.4
2011-06-27 09:45:31-0700 [-] recording hostname in twistd.hostname

The buildslave took more than 10 seconds to start, so we were unable to
confirm that it started correctly. Please 'tail twistd.log' and look for a
line that says 'configuration update complete' to verify correct startup.

$

Now, I thought of updating Twisted, but the Twisted downloads page says,

``Twisted is no longer making a .dmg for OS X because OS X versions 10.5
and greater ship with Twisted installed and the default Python path
configuration makes it impossible for a user to upgrade Twisted without
possibly breaking applications which depend on the system Twisted
installation.''

Anyone know just what those applications are?  All this machine is doing
is running the Python buildbot.

Bill

From fdrake at acm.org  Mon Jun 27 19:02:22 2011
From: fdrake at acm.org (Fred Drake)
Date: Mon, 27 Jun 2011 13:02:22 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <25745.1309189908@parc.com>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E089340.7060004@voidspace.org.uk>
	<BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>
	<25745.1309189908@parc.com>
Message-ID: <BANLkTi=D=EN3Ub-crBxzxFE=er=zBiXGPg@mail.gmail.com>

Egads.

Back when I wrote

    "Members and methods" should just be "attributes".

I used quotes to specifically indicate that this applied to the phrase
"members and methods", not their separate use.  I guess I wasn't obvious
enough.

The general & Python-historical uses of "members" is unfortunate.

My current position on this is that we should avoid the use of "members",
because either use will confuse a large set of readers.

As Nick points out, these are all attributes, regardless of their
implementation or type of the value.  "Methods" is a convenient and widely
understood term to refer to a general class of attributes, when that actually
matches the meaning.

For non-method or not-necessarily-a-method attributes, I'm inclined to just
stick with calling them attributes at this point.

Even more important, we need to decide what to call them, and add appropriate
words to the glossary.  And then make the documentation match that.


  -Fred

-- 
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"Give me the luxuries of life and I will willingly do without the necessities."
?? --Frank Lloyd Wright

From fuzzyman at voidspace.org.uk  Mon Jun 27 19:10:52 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 27 Jun 2011 18:10:52 +0100
Subject: [Python-Dev] Snow Leopard buildbot failing again...
In-Reply-To: <26766.1309194073@parc.com>
References: <26766.1309194073@parc.com>
Message-ID: <4E08B99C.5040807@voidspace.org.uk>

On 27/06/2011 18:01, Bill Janssen wrote:
> I see that parc-snowleopard-1 went down again.  I've done a software
> update, rebooted, and installed the latest buildslave, 0.8.4.  I can
> ping dinsdale.python.org successfully from the machine.  However, when I
> start the buildslave, I get this:
>
> $ buildslave start ~/buildarea/
> /Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/internet/_sslverify.py:5: DeprecationWarning: the md5 module is deprecated; use hashlib instead
>    import itertools, md5
> Following twistd.log until startup finished..
> /Library/Python/2.6/site-packages/buildbot_slave-0.8.4-py2.6.egg/buildslave/scripts/logwatcher.py:67: PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process.
>    env=os.environ,
> /Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/persisted/sob.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead
>    import os, md5, sys
> /Library/Python/2.6/site-packages/Twisted-8.2.0-py2.6-macosx-10.6-universal.egg/twisted/python/filepath.py:12: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
>    import sha
> 2011-06-27 09:45:31-0700 [-] Log opened.
> 2011-06-27 09:45:31-0700 [-] twistd 8.2.0 (/usr/bin/python 2.6.1) starting up.
> 2011-06-27 09:45:31-0700 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
> 2011-06-27 09:45:31-0700 [-] Starting BuildSlave -- version: 0.8.4
> 2011-06-27 09:45:31-0700 [-] recording hostname in twistd.hostname
>
> The buildslave took more than 10 seconds to start, so we were unable to
> confirm that it started correctly. Please 'tail twistd.log' and look for a
> line that says 'configuration update complete' to verify correct startup.
>
> $
>
> Now, I thought of updating Twisted, but the Twisted downloads page says,
>
> ``Twisted is no longer making a .dmg for OS X because OS X versions 10.5
> and greater ship with Twisted installed and the default Python path
> configuration makes it impossible for a user to upgrade Twisted without
> possibly breaking applications which depend on the system Twisted
> installation.''
>
> Anyone know just what those applications are?  All this machine is doing
> is running the Python buildbot.

Well the calendar server (at least) uses the system twisted. It is best 
to leave the system python alone and install / use a python.org python 
install. Are the buildbot scripts hardcoded to use the system python on 
Macs? If so you will have to stick with the system version of twisted, 
which seems to be problematic.

All the best,

Michael

> Bill
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From solipsis at pitrou.net  Mon Jun 27 19:27:26 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 27 Jun 2011 19:27:26 +0200
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E089340.7060004@voidspace.org.uk>
	<BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>
Message-ID: <20110627192726.51455b5d@msiwind>

Le Tue, 28 Jun 2011 00:36:20 +1000,
Nick Coghlan <ncoghlan at gmail.com> a ?crit :
> And no, the fact that methods can be treated as attributes is not a
> minor detail. It is *fundamental* to Python's object model that
> *methods are not a special case of attribute access*.

Uh, and so what?

Again, the stdlib docs are describing APIs from a high-level viewpoint, not
the Python object model. Just because a method is treated like other
attributes, or a function is treated like other callable objects, doesn't
mean we should replace "method" with "attribute", or "function" with
"callable object", or "class" with "instance of the type type".

I'm -1 on such a change anyway.

Regards

Antoine.

From exarkun at twistedmatrix.com  Mon Jun 27 19:36:41 2011
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Mon, 27 Jun 2011 17:36:41 -0000
Subject: [Python-Dev] Snow Leopard buildbot failing again...
In-Reply-To: <26766.1309194073@parc.com>
References: <26766.1309194073@parc.com>
Message-ID: <20110627173641.1894.764887772.divmod.xquotient.459@localhost.localdomain>

On 05:01 pm, janssen at parc.com wrote:
>I see that parc-snowleopard-1 went down again.  I've done a software
>update, rebooted, and installed the latest buildslave, 0.8.4.  I can
>ping dinsdale.python.org successfully from the machine.  However, when 
>I
>start the buildslave, I get this:
>
>[snip]

>
>The buildslave took more than 10 seconds to start, so we were unable to
>confirm that it started correctly. Please 'tail twistd.log' and look 
>for a
>line that says 'configuration update complete' to verify correct 
>startup.

Did you look at twistd.log?

Jean-Paul

From tjreedy at udel.edu  Mon Jun 27 20:33:38 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 27 Jun 2011 14:33:38 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <20110627102428.6fc1955b@msiwind>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
Message-ID: <iuaie1$qlm$1@dough.gmane.org>

On 6/27/2011 4:24 AM, Antoine Pitrou wrote:
> Le Mon, 27 Jun 2011 11:32:32 +1000,
> Nick Coghlan<ncoghlan at gmail.com>  a ?crit :
>

>> "Members" is a historical relic that is best replaced by "attributes"
>> or "data attributes" if we want to explicitly exclude methods for some
>> reason. "Methods" is a subset of attributes that explicitly excludes
>> data attributes.
>
> While I know it is technically right, I find it a bit strange to refer to
> methods as "attributes". We're describing an API, not the inner working of
> the object model. Also, people just discovering Python will probably be a
> bit surprised if we start refer to methods as "attributes".
>
> FWIW, I tend to understand "members" as "methods + attributes", which makes
> it a nice term to use for that purpose.

Let me repeat that that is historically wrong for Python, and illustrate 
why the term 'members' should not be used. From the 1.5 Language 
Reference, 3.2 Standard type hierarchy: "There are also some 'generic' 
special attributes, not listed with the individual objects: __methods__ 
is a list of the method names of a built-in object, if it has any; 
__members__ is a list of the data attribute names of a built-in object, 
if it has any."

-- 
Terry Jan Reedy



From janssen at parc.com  Mon Jun 27 21:00:09 2011
From: janssen at parc.com (Bill Janssen)
Date: Mon, 27 Jun 2011 12:00:09 PDT
Subject: [Python-Dev] Snow Leopard buildbot failing again...
In-Reply-To: <20110627173641.1894.764887772.divmod.xquotient.459@localhost.localdomain>
References: <26766.1309194073@parc.com>
	<20110627173641.1894.764887772.divmod.xquotient.459@localhost.localdomain>
Message-ID: <29834.1309201209@parc.com>

exarkun at twistedmatrix.com wrote:

> On 05:01 pm, janssen at parc.com wrote:
> >I see that parc-snowleopard-1 went down again.  I've done a software
> >update, rebooted, and installed the latest buildslave, 0.8.4.  I can
> > ping dinsdale.python.org successfully from the machine.  However,
> > when I
> >start the buildslave, I get this:
> >
> >[snip]
> 
> >
> >The buildslave took more than 10 seconds to start, so we were unable to
> > confirm that it started correctly. Please 'tail twistd.log' and look
> > for a
> > line that says 'configuration update complete' to verify correct
> > startup.
> 
> Did you look at twistd.log?
> 
> Jean-Paul

Um... not to look for "configuration update complete", I'm afraid.
Isn't that a buildmaster-only thing?  Here's what was there.

2011-06-27 09:45:31-0700 [-] Log opened.
2011-06-27 09:45:31-0700 [-] twistd 8.2.0 (/usr/bin/python 2.6.1) starting up.
2011-06-27 09:45:31-0700 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2011-06-27 09:45:31-0700 [-] Starting BuildSlave -- version: 0.8.4
2011-06-27 09:45:31-0700 [-] recording hostname in twistd.hostname
2011-06-27 09:46:01-0700 [-] Starting factory <buildslave.bot.BotFactory instance at 0x101637368>
2011-06-27 09:46:01-0700 [-] Connecting to dinsdale.python.org:9020
2011-06-27 09:46:02-0700 [Broker,client] message from master: attached
2011-06-27 09:46:02-0700 [Broker,client] I have a leftover directory '3.1.parc-snowleopard-1' that is not being used by the buildmaster: you can delete it now
2011-06-27 09:46:03-0700 [Broker,client] SlaveBuilder.remote_print(AMD64 Snow Leopard 2 custom): message from master: attached
2011-06-27 09:46:03-0700 [Broker,client] SlaveBuilder.remote_print(AMD64 Snow Leopard 2 2.7): message from master: attached
2011-06-27 09:46:03-0700 [Broker,client] SlaveBuilder.remote_print(AMD64 Snow Leopard 2 3.x): message from master: attached
2011-06-27 09:46:03-0700 [Broker,client] SlaveBuilder.remote_print(AMD64 Snow Leopard 2 3.2): message from master: attached
2011-06-27 09:46:05-0700 [Broker,client] Connected to dinsdale.python.org:9020; slave is ready
2011-06-27 09:46:05-0700 [Broker,client] sending application-level keepalives every 600 seconds

There is that 30-second lag between "recording hostname in
twistd.hostname" and "Starting factory", which is presumably what
triggers the warning from buildslave.  See
<http://trac.buildbot.net/ticket/68> and <http://trac.buildbot.net/ticket/745>.

Bill

From janssen at parc.com  Mon Jun 27 21:05:38 2011
From: janssen at parc.com (Bill Janssen)
Date: Mon, 27 Jun 2011 12:05:38 PDT
Subject: [Python-Dev] Snow Leopard buildbot failing again...
In-Reply-To: <20110627173641.1894.764887772.divmod.xquotient.459@localhost.localdomain>
References: <26766.1309194073@parc.com>
	<20110627173641.1894.764887772.divmod.xquotient.459@localhost.localdomain>
Message-ID: <30385.1309201538@parc.com>

I also find <http://trac.buildbot.net/ticket/1854> interesting, because
it seems a good explanation of why the build slave might go into the
zombie state of attempting to reconnect to the master.

Bill

From rdmurray at bitdance.com  Mon Jun 27 21:22:16 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 27 Jun 2011 15:22:16 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <20110627192726.51455b5d@msiwind>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E089340.7060004@voidspace.org.uk>
	<BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>
	<20110627192726.51455b5d@msiwind>
Message-ID: <20110627192217.66A3F2505A3@webabinitio.net>

On Mon, 27 Jun 2011 19:27:26 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Tue, 28 Jun 2011 00:36:20 +1000,
> Nick Coghlan <ncoghlan at gmail.com> a ??crit :
> > And no, the fact that methods can be treated as attributes is not a
> > minor detail. It is *fundamental* to Python's object model that
> > *methods are not a special case of attribute access*.
> 
> Uh, and so what?
> 
> Again, the stdlib docs are describing APIs from a high-level viewpoint, not
> the Python object model. Just because a method is treated like other
> attributes, or a function is treated like other callable objects, doesn't
> mean we should replace "method" with "attribute", or "function" with
> "callable object", or "class" with "instance of the type type".

I don't think there is any disagreement with regards to replacing
'methods' with 'attributes'.  This should not be done, in general, and
I don't think anybody involved in this particular discussion thinks it
should be.  The question is what to call non-method attributes, or how we
refer to a mix of method and non-method attributes.  In some cases just
saying 'attributes' may make sense for the latter, but in most cases it 
probably doesn't, because many people coming from other languages do
not expect methods to be first class objects.  So I'd like to be able
to say, in those cases, "attributes (XXXs and methods)".

The subject of this thread is the removal of the use of the term 'members'
for XXX, for the historical reasons cited.

IMO the goal should be to help the reader's mental model match what Python
actually does.  In that sense Python's object model *does* matter[*],
in that we shouldn't make it harder for the reader to figure out what is
"really" going on.  In the cases you cited, you are arguing that the
more specific term (method, function, class) should not be replaced
in the docs by the more generic term (attribute, callable object,
instance of type type).  I don't think anyone would argue with you
about that in general.  (In specific there might in fact be some
places in the docs where such a change would improve clarity!)

So, the correct generic term for something that can be accessed
via attribute notation is attribute.  The more specific term for an
attribute that is a method is method.  We don't currently have a more
specific collective term for attributes that aren't methods.  *That*
is the problem.

--
R. David Murray           http://www.bitdance.com

[*] it also seems to me that the object model is, in many ways, *part*
of the API, in the sense that the API is how you access the object model,
and so understanding the object model is, if not essentially, then at
least very helpful.

From fuzzyman at voidspace.org.uk  Mon Jun 27 21:30:12 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 27 Jun 2011 20:30:12 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <20110627192217.66A3F2505A3@webabinitio.net>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E089340.7060004@voidspace.org.uk>	<BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>	<20110627192726.51455b5d@msiwind>
	<20110627192217.66A3F2505A3@webabinitio.net>
Message-ID: <4E08DA44.5010507@voidspace.org.uk>

On 27/06/2011 20:22, R. David Murray wrote:
> [snip...]
> So, the correct generic term for something that can be accessed
> via attribute notation is attribute.  The more specific term for an
> attribute that is a method is method.  We don't currently have a more
> specific collective term for attributes that aren't methods.  *That*
> is the problem.

I think part of the problem is also that *some* people's usage of the 
term doesn't match the precise terminology.

It sounds like if I say "an object's attributes" some of us will expect 
that to include the methods and some of us not. I'd say anecdotally that 
when people talk about object attributes collectively they are *not* 
including methods. When they talk about attribute lookup that rightly 
includes everything.

We do have other terms: instance attributes (which do not include 
methods) and class attributes (which technically do - but is also used 
to indicate attributes set on the class rather than the instance but not 
including methods [1]).

A precise term to describe "attributes that are not methods" would still 
be helpful. I guess the closest we have is "non-descriptors", but that 
requires a knowledge of the descriptor protocol for it to be useful.

Making-things-worse-and-not-better'ly yours,

Michael

[1] I'm talking about *usage* of the term here... I guess usage is 
inconsistent anyway, sometimes people will mean to include methods and 
sometimes not.
> --
> R. David Murray           http://www.bitdance.com
>
> [*] it also seems to me that the object model is, in many ways, *part*
> of the API, in the sense that the API is how you access the object model,
> and so understanding the object model is, if not essentially, then at
> least very helpful.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110627/973dcc44/attachment.html>

From rdmurray at bitdance.com  Mon Jun 27 22:37:21 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 27 Jun 2011 16:37:21 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <4E08DA44.5010507@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E089340.7060004@voidspace.org.uk>
	<BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>
	<20110627192726.51455b5d@msiwind>
	<20110627192217.66A3F2505A3@webabinitio.net>
	<4E08DA44.5010507@voidspace.org.uk>
Message-ID: <20110627203721.C6F7F2505A3@webabinitio.net>

On Mon, 27 Jun 2011 20:30:12 +0100, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> On 27/06/2011 20:22, R. David Murray wrote:
> > [snip...]
> > So, the correct generic term for something that can be accessed
> > via attribute notation is attribute.  The more specific term for an
> > attribute that is a method is method.  We don't currently have a more
> > specific collective term for attributes that aren't methods.  *That*
> > is the problem.
> 
> I think part of the problem is also that *some* people's usage of the 
> term doesn't match the precise terminology.
> 
> It sounds like if I say "an object's attributes" some of us will expect 
> that to include the methods and some of us not. I'd say anecdotally that 
> when people talk about object attributes collectively they are *not* 
> including methods. When they talk about attribute lookup that rightly 
> includes everything.

Yes, I think that it is common to use 'attributes' to mean 'non-method
attributes', and often the context can make clear which usage is
intended.  What we should do in our own docs is a different question,
but we do have other cases where "it is clear from context" is
considered a valid argument for technically imprecise terminology.
And I think that's reasonable.

> A precise term to describe "attributes that are not methods" would still 
> be helpful. I guess the closest we have is "non-descriptors", but that 
> requires a knowledge of the descriptor protocol for it to be useful.

I'm not really as comfortable as I think I should be with the descriptor
stuff, but aren't properties descriptors in the sense you are using it
here?  Yet I don't think a property should be called a method, even though
it is implemented via a function(s) that looks a lot like a method.  This is
why talking about non-method attributes collectively is so fraught....

--
R. David Murray           http://www.bitdance.com

From tjreedy at udel.edu  Tue Jun 28 00:05:33 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 27 Jun 2011 18:05:33 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <4E08509E.1030401@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic>	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>
	<4E08509E.1030401@voidspace.org.uk>
Message-ID: <iuaurc$6qq$1@dough.gmane.org>

On 6/27/2011 5:42 AM, Michael Foord wrote:
> On 27/06/2011 09:24, Antoine Pitrou wrote:

>> FWIW, I tend to understand "members" as "methods + attributes", which
>> makes
>> it a nice term to use for that purpose.
>
> That is my understanding / use of the terms as well.

On 6/27/2011 5:45 AM, Oleg Broytman wrote:
 >     That's my feeling too.

Whereas the actual existing usage is that attributes = members + 
methods. 'Member' came from 'member of a data structure', as in complex 
numbers have real and imag members, whereas lists have methods but no 
(user visible) members.

The fact that so many people get the Python usage of 'member' so wrong 
is a good reason to retire it.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Tue Jun 28 00:18:29 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 27 Jun 2011 18:18:29 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <iuaie1$qlm$1@dough.gmane.org>
References: <20110626185242.GB2710@mathmagic>	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>
	<iuaie1$qlm$1@dough.gmane.org>
Message-ID: <iuavjj$9ga$1@dough.gmane.org>

On 6/27/2011 2:33 PM, Terry Reedy wrote:

> Let me repeat that that is historically wrong for Python, and illustrate
> why the term 'members' should not be used. From the 1.5 Language
> Reference, 3.2 Standard type hierarchy: "There are also some 'generic'
> special attributes, not listed with the individual objects: __methods__
> is a list of the method names of a built-in object, if it has any;
> __members__ is a list of the data attribute names of a built-in object,
> if it has any."

This sentence was left untouched until 2.2. What's new 2.2 has "In 
previous versions of Python, there was no consistent way to discover 
what attributes and methods were supported by an object. There were some 
informal conventions, such as defining __members__ and __methods__ 
attributes that were lists of names, but often the author of an 
extension type or a class wouldn't bother to define them."
This is a section on descriptors, but the real replacement is, I 
believe, dir().

-- 
Terry Jan Reedy


From fuzzyman at voidspace.org.uk  Tue Jun 28 00:46:53 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 27 Jun 2011 23:46:53 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <iuavjj$9ga$1@dough.gmane.org>
References: <20110626185242.GB2710@mathmagic>	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<iuaie1$qlm$1@dough.gmane.org>
	<iuavjj$9ga$1@dough.gmane.org>
Message-ID: <4E09085D.80202@voidspace.org.uk>

On 27/06/2011 23:18, Terry Reedy wrote:
> On 6/27/2011 2:33 PM, Terry Reedy wrote:
>
>> Let me repeat that that is historically wrong for Python, and illustrate
>> why the term 'members' should not be used. From the 1.5 Language
>> Reference, 3.2 Standard type hierarchy: "There are also some 'generic'
>> special attributes, not listed with the individual objects: __methods__
>> is a list of the method names of a built-in object, if it has any;
>> __members__ is a list of the data attribute names of a built-in object,
>> if it has any."
>
> This sentence was left untouched until 2.2. What's new 2.2 has "In 
> previous versions of Python, there was no consistent way to discover 
> what attributes and methods were supported by an object. There were 
> some informal conventions, such as defining __members__ and 
> __methods__ attributes that were lists of names, but often the author 
> of an extension type or a class wouldn't bother to define them."
So the Python 2.2 what's new talks about attributes and methods as 
different things.... Of course the context makes it clear, but this 
mirrors how I use the terms in discussion and how I see others generally 
using them.

Great topic for bikeshedding. :-)

Michael

> This is a section on descriptors, but the real replacement is, I 
> believe, dir().
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From guido at python.org  Tue Jun 28 01:32:16 2011
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Jun 2011 16:32:16 -0700
Subject: [Python-Dev] pdb and generators
Message-ID: <BANLkTinkTWS_hMH3TMCripo3nmtR6MV=4A@mail.gmail.com>

When debugging generators (especially when using them as coroutines) I
often want 'next' to step over a yield statement/expression, instead
of the current behavior, which is to step out of the frame, since pdb
doesn't seem to see the difference between yield and return. Or I
could live with a separate command to step over the yield. Has anyone
looked into this yet? I found a question about this on
StackOverflow.com but the answers were particularly unhelpful, so I'm
guessing I'm not the first to want this but it hasn't been solved yet.
It might be a good project for someone looking into hacking pdb.

-- 
--Guido van Rossum (python.org/~guido)

From ncoghlan at gmail.com  Tue Jun 28 02:10:36 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2011 10:10:36 +1000
Subject: [Python-Dev] PEP 380 acceptance (was Re: EuroPython Language
 Summit report)
In-Reply-To: <loom.20110627T170547-215@post.gmane.org>
References: <BANLkTin1Voeg-7hBUOSwrMQNwu9BqE-nQA@mail.gmail.com>
	<BANLkTikzx4w3GS9FfHM+q4GQ1Ur-bU5L_A@mail.gmail.com>
	<loom.20110627T170547-215@post.gmane.org>
Message-ID: <BANLkTimMMg+DHbkuXFdVtqiRcCFVL06dew@mail.gmail.com>

On Tue, Jun 28, 2011 at 1:09 AM, renaud <rndblnch at gmail.com> wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
>> I hit a snag with this. The real tests of the PEP 380 functionality
>> aren't currently part of the patch - they're a big set of "golden
>> output" tests in the zipfile hosted on Greg's site. Those need to be
>> refactored into proper unittest or doctest based additions to the test
>> suite and incorporated into the patch before I could commit this with
>> a clear conscience.
>
> let me know if i can help.

It would be good if you could take a look at Greg's original test
suite, consider ways of bringing it into the main regression tests and
then update the patch queue on bitbucket accordingly.

My preference is for something unittest based, essentially taking the
"golden output" comparisons and turning them into appropriate
self.assert* invocations.

Given the number of tests Greg has, it will probably make more sense
to do it as a new test subdirectory rather than as a single test file
(although that depends on how many tests are in each file - if there
are only a few, or if they overlap a lot, then having them as separate
test cases within a single file may be a better choice).

>> Renaud's patch mostly applies cleanly at the moment - the only change
>> is that the "#endif" for the Py_LIMITED_API check needs to be moved in
>> pyerrors.h so it also covers the new StopIteration struct definition.
>
> if this helps, i've updated the patch to fix this.
> https://bitbucket.org/rndblnch/cpython-pep380/changeset/6014d1720625

Yep, that does help.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From steve at pearwood.info  Tue Jun 28 02:14:57 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 28 Jun 2011 10:14:57 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <20110627192217.66A3F2505A3@webabinitio.net>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E089340.7060004@voidspace.org.uk>	<BANLkTimGUn-gBce3eEKddWFoi6rcxXrj5A@mail.gmail.com>	<20110627192726.51455b5d@msiwind>
	<20110627192217.66A3F2505A3@webabinitio.net>
Message-ID: <4E091D01.4090105@pearwood.info>

R. David Murray wrote:

> So, the correct generic term for something that can be accessed
> via attribute notation is attribute.  The more specific term for an
> attribute that is a method is method.  We don't currently have a more
> specific collective term for attributes that aren't methods.  *That*
> is the problem.

"Attribute" also encompasses both instance attributes and class 
attributes. Rather than having two different words, we simply qualify 
the word when we need to distinguish them.

Likewise, in the cases where it is important to distinguish methods from 
other attributes, we should qualify the word: data attribute vs method 
attribute.

(I'm not suggesting that we should routinely refer to "method attribute" 
rather than simply method, but only when we wish to emphasize that 
methods are a kind of attribute and not a completely different kind of 
thing.)



-- 
Steven

From ncoghlan at gmail.com  Tue Jun 28 02:30:14 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2011 10:30:14 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09085D.80202@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind> <iuaie1$qlm$1@dough.gmane.org>
	<iuavjj$9ga$1@dough.gmane.org> <4E09085D.80202@voidspace.org.uk>
Message-ID: <BANLkTikJGTxQ9Pv-Y7dkHVGNrSdago1ZyA@mail.gmail.com>

On Tue, Jun 28, 2011 at 8:46 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> So the Python 2.2 what's new talks about attributes and methods as different
> things.... Of course the context makes it clear, but this mirrors how I use
> the terms in discussion and how I see others generally using them.
>
> Great topic for bikeshedding. :-)

Yep, as David said, "attribute" is legitimately used to mean *both*
"all attributes (i.e. both data attributes and methods)" and "data
attributes (i.e. excluding methods)". In general, context makes it
clear which meaning is intended, and when that isn't the case, more
verbose phrasing such as that in the previous sentence can make it
explicit.

Rather than fighting that convention, we should probably just confront
the ambiguity head on and update
http://docs.python.org/dev/glossary.html#term-attribute to describe
both uses of the term (and add a separate entry for "data attribute",
with a definition which basically says "attributes which are not
methods").

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ben+python at benfinney.id.au  Tue Jun 28 04:11:18 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 28 Jun 2011 12:11:18 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com>
Message-ID: <87r56en1rt.fsf@benfinney.id.au>

Rob Cliffe <rob.cliffe at btinternet.com> writes:

> On 27/06/2011 15:08, R. David Murray wrote:

> > I guess what I'm saying is that I am more comfortable calling them
> > all attributes than calling them all members. The term 'members'
> > isn't used anywhere in the language itself, as far as I can recall,
> > whereas getattr and setattr are evidence that the language considers
> > them all attributes. I think we do the documentation readers a
> > disservice by obscuring that fact by using other terminology.

+1

> 'function attributes' ?  'def attributes' ?

?1. They don't have to be functions, and hence don't have to be created
by ?def?.

> Or just stick with method attributes' ?

?callable attributes? describes exactly what they are, in terms that
will remain useful to the person learning Python.

-- 
 \          ??Did you sleep well?? ?No, I made a couple of mistakes.?? |
  `\                                                    ?Steven Wright |
_o__)                                                                  |
Ben Finney


From ericsnowcurrently at gmail.com  Tue Jun 28 05:47:12 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Mon, 27 Jun 2011 21:47:12 -0600
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <87r56en1rt.fsf@benfinney.id.au>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
Message-ID: <BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>

On Mon, Jun 27, 2011 at 8:11 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> Rob Cliffe <rob.cliffe at btinternet.com> writes:
>
>> 'function attributes' ?  'def attributes' ?
>
> -1. They don't have to be functions, and hence don't have to be created
> by 'def'.
>
>> Or just stick with method attributes' ?
>
> "callable attributes" describes exactly what they are, in terms that
> will remain useful to the person learning Python.
>

The usage of the object determines what we call it then, so what about
"state attributes" in the same vein as "callable attributes" (data vs.
method).  But it would be nice to have the names consistent across the
different contexts.

ABCMeta tests __isabstractmethod__ on each attribute of a class, not
just the methods, rather that __isabstractattribute__.  Perhaps
calling something a method attribute even when it isn't a function is
still okay.  Thus the pair could be "method and data attributes".
"method attribute" is a little redundant but calling it a "function
attribute" seems less consistent

-eric

> --
>  \          "'Did you sleep well?' 'No, I made a couple of mistakes.'" |
>  `\                                                    --Steven Wright |
> _o__)                                                                  |
> Ben Finney
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>

From ben+python at benfinney.id.au  Tue Jun 28 07:05:03 2011
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 28 Jun 2011 15:05:03 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
Message-ID: <87mxh2mtq8.fsf@benfinney.id.au>

Eric Snow <ericsnowcurrently at gmail.com> writes:

> On Mon, Jun 27, 2011 at 8:11 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> > "callable attributes" describes exactly what they are, in terms that
> > will remain useful to the person learning Python.
>
> The usage of the object determines what we call it then

No, the capability of the attribute is what determines that difference,
not how it's used.

I still don't have a good term for ?non-callable attribute?, though.

-- 
 \         ?I'm not a bad guy! I work hard, and I love my kids. So why |
  `\      should I spend half my Sunday hearing about how I'm going to |
_o__)                                            Hell?? ?Homer Simpson |
Ben Finney


From martin at v.loewis.de  Tue Jun 28 08:21:29 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 28 Jun 2011 08:21:29 +0200
Subject: [Python-Dev] Snow Leopard buildbot failing again...
In-Reply-To: <30385.1309201538@parc.com>
References: <26766.1309194073@parc.com>	<20110627173641.1894.764887772.divmod.xquotient.459@localhost.localdomain>
	<30385.1309201538@parc.com>
Message-ID: <4E0972E9.5080400@v.loewis.de>

Am 27.06.2011 21:05, schrieb Bill Janssen:
> I also find <http://trac.buildbot.net/ticket/1854> interesting, because
> it seems a good explanation of why the build slave might go into the
> zombie state of attempting to reconnect to the master.

That wouldn't apply to our buildslaves, though, since we don't have
FileUpload build steps (except for a single builder run by David Bolen).

Regards,
Martin

From ncoghlan at gmail.com  Tue Jun 28 08:33:23 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2011 16:33:23 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <87mxh2mtq8.fsf@benfinney.id.au>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
Message-ID: <BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>

On Tue, Jun 28, 2011 at 3:05 PM, Ben Finney <ben+python at benfinney.id.au> wrote:
> I still don't have a good term for ?non-callable attribute?, though.

The two terms I've got out of this thread are "callable attributes"
(instance/static/class methods, etc) and "data attributes" (everything
else). Both seem reasonable to me, creating two largely disjoint sets
that together cover all the different kinds of attribute you're likely
to encounter.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From nad at acm.org  Tue Jun 28 10:17:16 2011
From: nad at acm.org (Ned Deily)
Date: Tue, 28 Jun 2011 01:17:16 -0700
Subject: [Python-Dev] cpython: remove duplicate -I
References: <E1QYQFe-0006Ir-1o@dinsdale.python.org>
Message-ID: <nad-67ECB0.01171628062011@news.gmane.org>

In article <E1QYQFe-0006Ir-1o at dinsdale.python.org>,
 benjamin.peterson <python-checkins at python.org> wrote:

> http://hg.python.org/cpython/rev/0d0ca6f95d9d
> changeset:   70879:0d0ca6f95d9d
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Sun Jun 19 17:17:30 2011 -0500
> summary:
>   remove duplicate -I
> 
> files:
>   Makefile.pre.in |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Makefile.pre.in b/Makefile.pre.in
> --- a/Makefile.pre.in
> +++ b/Makefile.pre.in
> @@ -73,7 +73,7 @@
>  # Both CPPFLAGS and LDFLAGS need to contain the shell's value for setup.py 
>  to
>  # be able to build extension modules using the directories specified in the
>  # environment variables
> -PY_CPPFLAGS=	-I. -IInclude -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) 
> $(CPPFLAGS)
> +PY_CPPFLAGS=	-I. -IInclude $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)
>  PY_LDFLAGS=	$(CONFIGURE_LDFLAGS) $(LDFLAGS)
>  NO_AS_NEEDED=	@NO_AS_NEEDED@
>  LDLAST=		@LDLAST@

I think someone has already mentioned that this change breaks builds 
with a source directory outside the build directory and that breaks my 
OS X installer builds.   Please revert or make the following change 
which works for me (but not tested elsewhere):

-PY_CPPFLAGS=   -I. -IInclude $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)
+PY_CPPFLAGS=   -I. -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)

-- 
 Ned Deily,
 nad at acm.org


From solipsis at pitrou.net  Tue Jun 28 10:24:31 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 28 Jun 2011 10:24:31 +0200
Subject: [Python-Dev] Snow Leopard buildbot failing again...
References: <26766.1309194073@parc.com>
Message-ID: <20110628102431.2fd841d4@pitrou.net>

On Mon, 27 Jun 2011 10:01:13 PDT
Bill Janssen <janssen at parc.com> wrote:
> I see that parc-snowleopard-1 went down again.  I've done a software
> update, rebooted, and installed the latest buildslave, 0.8.4.  I can
> ping dinsdale.python.org successfully from the machine.  However, when I
> start the buildslave, I get this:

Well, doesn't the slave work correctly now? Or is there a problem?




From fdrake at acm.org  Tue Jun 28 12:44:08 2011
From: fdrake at acm.org (Fred Drake)
Date: Tue, 28 Jun 2011 06:44:08 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
Message-ID: <BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>

On Tue, Jun 28, 2011 at 2:33 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> The two terms I've got out of this thread are "callable attributes"
> (instance/static/class methods, etc) and "data attributes" (everything
> else). Both seem reasonable to me, creating two largely disjoint sets
> that together cover all the different kinds of attribute you're likely
> to encounter.

But "callable attributes" aren't the same thing as methods; most are methods,
but not all.  Sometimes, they're data used by the object.  The fact that
data attributes can be callable is irrelevant.


  -Fred

-- 
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"Give me the luxuries of life and I will willingly do without the necessities."
?? --Frank Lloyd Wright

From fuzzyman at voidspace.org.uk  Tue Jun 28 12:54:39 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 11:54:39 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>
	<87r56en1rt.fsf@benfinney.id.au>	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>	<87mxh2mtq8.fsf@benfinney.id.au>	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
Message-ID: <4E09B2EF.9030304@voidspace.org.uk>

On 28/06/2011 11:44, Fred Drake wrote:
> On Tue, Jun 28, 2011 at 2:33 AM, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>> The two terms I've got out of this thread are "callable attributes"
>> (instance/static/class methods, etc) and "data attributes" (everything
>> else). Both seem reasonable to me, creating two largely disjoint sets
>> that together cover all the different kinds of attribute you're likely
>> to encounter.
> But "callable attributes" aren't the same thing as methods; most are methods,
> but not all.  Sometimes, they're data used by the object.  The fact that
> data attributes can be callable is irrelevant.

Added to which there are other descriptors, notably property, that are 
not directly callable but are not provided as normal "data attributes" 
(although the access syntax is the same). Properties are much closer to 
methods as they are implemented on the class and fetched via the 
descriptor protocol. Instead of "data attributes" I prefer the term 
"instance attributes" although that doesn't include "class attributes" 
(or more precisely it doesn't cover "class attributes that aren't 
descriptors").

The problem with "data attributes" is that it doesn't mean *anything*, 
which I suppose is useful for invented terminology, but it means it 
doesn't convey anything precise to those who haven't heard the term 
before. If it becomes widely used then that changes I guess. I'd still 
normally just use "attributes" though...

All the best,

Michael Foord
>
>    -Fred
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From fdrake at acm.org  Tue Jun 28 13:04:11 2011
From: fdrake at acm.org (Fred Drake)
Date: Tue, 28 Jun 2011 07:04:11 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09B2EF.9030304@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
Message-ID: <BANLkTimC6=OAWHLK7_XU+teUCQmugA=zgw@mail.gmail.com>

On Tue, Jun 28, 2011 at 6:54 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Added to which there are other descriptors, notably property, that are not
> directly callable but are not provided as normal "data attributes" (although
> the access syntax is the same). Properties are much closer to methods as
> they are implemented on the class and fetched via the descriptor protocol.
> Instead of "data attributes" I prefer the term "instance attributes"
> although that doesn't include "class attributes" (or more precisely it
> doesn't cover "class attributes that aren't descriptors").

Given the availability of __getattr__ and __getattribute__, I consider
properties an implementation detail for some attributes.  The fact that
Python code is called on access is only marginally interesting.

> The problem with "data attributes" is that it doesn't mean *anything*, which
> I suppose is useful for invented terminology, but it means it doesn't convey
> anything precise to those who haven't heard the term before. If it becomes
> widely used then that changes I guess. I'd still normally just use
> "attributes" though...

I'd read "data attributes" the same as "non-method attributes".  For readers,
calling them "attributes" is typically sufficient.  It's rare to need to
distinguish them from methods.


  -Fred

-- 
Fred L. Drake, Jr.? ? <fdrake at acm.org>
"Give me the luxuries of life and I will willingly do without the necessities."
?? --Frank Lloyd Wright

From fuzzyman at voidspace.org.uk  Tue Jun 28 13:35:40 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 12:35:40 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <BANLkTimC6=OAWHLK7_XU+teUCQmugA=zgw@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
	<BANLkTimC6=OAWHLK7_XU+teUCQmugA=zgw@mail.gmail.com>
Message-ID: <4E09BC8C.7090807@voidspace.org.uk>

On 28/06/2011 12:04, Fred Drake wrote:
> On Tue, Jun 28, 2011 at 6:54 AM, Michael Foord
> <fuzzyman at voidspace.org.uk>  wrote:
>> Added to which there are other descriptors, notably property, that are not
>> directly callable but are not provided as normal "data attributes" (although
>> the access syntax is the same). Properties are much closer to methods as
>> they are implemented on the class and fetched via the descriptor protocol.
>> Instead of "data attributes" I prefer the term "instance attributes"
>> although that doesn't include "class attributes" (or more precisely it
>> doesn't cover "class attributes that aren't descriptors").
> Given the availability of __getattr__ and __getattribute__, I consider
> properties an implementation detail for some attributes.  The fact that
> Python code is called on access is only marginally interesting.
>

Well, they're *usually* implemented as methods and backed by a real 
instance attribute. Usually (but not always) it makes more sense (IME) 
to group them with methods. The fact that they're *accessed* as an 
attribute is the uninteresting detail.

__getattr__ and __getattribute__ are interesting - they allow you to use 
attribute access syntax for things that *aren't* attributes.

I appreciate the fact that the Python data-model means methods are just 
object attributes, but they're not *instance attributes* and sometimes 
you need to make a distinction. (And yes it is helpful if the standard 
terminology leads people into a better understanding of the Python data 
model, but that still doesn't change the need on occasion for 
terminology that doesn't need to be explained whenever it is used.)

Given that how even methods are to be described depends on the context 
(if you're fetching bound methods as objects then it makes perfect sense 
to just talk about them as attributes) it doesn't seem an area amenable 
to one-size-fits-all terminology.

>> The problem with "data attributes" is that it doesn't mean *anything*, which
>> I suppose is useful for invented terminology, but it means it doesn't convey
>> anything precise to those who haven't heard the term before. If it becomes
>> widely used then that changes I guess. I'd still normally just use
>> "attributes" though...
> I'd read "data attributes" the same as "non-method attributes".  For readers,
> calling them "attributes" is typically sufficient.  It's rare to need to
> distinguish them from methods.

Yeah, this is all a grand bikeshed. I'm not sure I would understand 
"data attributes" unless it was clear from the context. I would wonder 
what qualifies something as "data".

It is an interesting question what terminology we should use in the 
documentation if we need to distinguish them, but I think that is still 
wandering away from the original question that was posed.

All the best,

Michael

>
>    -Fred
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From rdmurray at bitdance.com  Tue Jun 28 13:51:35 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 28 Jun 2011 07:51:35 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <4E09B2EF.9030304@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
Message-ID: <20110628115136.248822505A3@webabinitio.net>

On Tue, 28 Jun 2011 11:54:39 +0100, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> On 28/06/2011 11:44, Fred Drake wrote:
> > But "callable attributes" aren't the same thing as methods; most are methods,
> > but not all.  Sometimes, they're data used by the object.  The fact that
> > data attributes can be callable is irrelevant.
> 
> Added to which there are other descriptors, notably property, that are 
> not directly callable but are not provided as normal "data attributes" 
> (although the access syntax is the same). Properties are much closer to 
> methods as they are implemented on the class and fetched via the 
> descriptor protocol. Instead of "data attributes" I prefer the term 
> "instance attributes" although that doesn't include "class attributes" 
> (or more precisely it doesn't cover "class attributes that aren't 
> descriptors").

Also, instances can have methods as instance attributes.

Trying to use 'instance attributes' for non-method attributes is a bad
idea, I think.

Given that there is no one thing that covers all non-method
attributes, I suspect 'non-method attributes' is as good as
we're going to manage.

--
R. David Murray           http://www.bitdance.com

From fuzzyman at voidspace.org.uk  Tue Jun 28 14:03:32 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 13:03:32 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <20110628115136.248822505A3@webabinitio.net>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
	<20110628115136.248822505A3@webabinitio.net>
Message-ID: <4E09C314.3090809@voidspace.org.uk>

On 28/06/2011 12:51, R. David Murray wrote:
> On Tue, 28 Jun 2011 11:54:39 +0100, Michael Foord<fuzzyman at voidspace.org.uk>  wrote:
>> On 28/06/2011 11:44, Fred Drake wrote:
>>> But "callable attributes" aren't the same thing as methods; most are methods,
>>> but not all.  Sometimes, they're data used by the object.  The fact that
>>> data attributes can be callable is irrelevant.
>> Added to which there are other descriptors, notably property, that are
>> not directly callable but are not provided as normal "data attributes"
>> (although the access syntax is the same). Properties are much closer to
>> methods as they are implemented on the class and fetched via the
>> descriptor protocol. Instead of "data attributes" I prefer the term
>> "instance attributes" although that doesn't include "class attributes"
>> (or more precisely it doesn't cover "class attributes that aren't
>> descriptors").
> Also, instances can have methods as instance attributes.
>
> Trying to use 'instance attributes' for non-method attributes is a bad
> idea, I think.

I would use instance attributes for members that are held in the 
instance dict (or have specific slots). As this can't be a normal 
"method" (it could be any object including a callable one - but won't be 
a standard method descriptor) it seems uncontroversial. (Or more to the 
point it seems to be *precise* in its meaning in the context of the 
Python data model.)

What do you mean by "instances can have methods as instance attributes"? 
Once you attach a bound method directly to an instance it becomes a 
slightly different beast I think. (On top of which that is pretty rare 
behaviour.)

> Given that there is no one thing that covers all non-method
> attributes, I suspect 'non-method attributes' is as good as
> we're going to manage.

Hehe, yeah - that's not bad... It's hard to see how it could be 
misunderstood too.

Michael

> --
> R. David Murray           http://www.bitdance.com


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From ncoghlan at gmail.com  Tue Jun 28 14:31:33 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2011 22:31:33 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09B2EF.9030304@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
Message-ID: <BANLkTimi_=T1AcPn-uMGYgnt90Lwag1OGA@mail.gmail.com>

On Tue, Jun 28, 2011 at 8:54 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> The problem with "data attributes" is that it doesn't mean *anything*, which
> I suppose is useful for invented terminology, but it means it doesn't convey
> anything precise to those who haven't heard the term before. If it becomes
> widely used then that changes I guess. I'd still normally just use
> "attributes" though...

In this context, precision is an illusion. There is *no* precise
terminology, not only because Python blurs the boundaries by design in
many respects (callable or not, method or not, attribute or not, etc),
but also because *people* use the same words to mean slightly
different things.

The best we can hope for is to encourage the right way of thinking
about the situation, and in that regard you have the complete set of
attributes accessible via an object (i.e. via __getattribute__), some
of which are callables (and may or may not act like instance methods)
and the remainder of which are data attributes (some of which may
incidentally be callable, even if they aren't used that way). How a
*particular* attribute is classified is not an inherent property of
the attribute, but also an artifact of the way it is used by the
application.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From tseaver at palladion.com  Tue Jun 28 14:43:35 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 28 Jun 2011 08:43:35 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>
	<87r56en1rt.fsf@benfinney.id.au>	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>	<87mxh2mtq8.fsf@benfinney.id.au>	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
Message-ID: <iuci9o$d6k$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/28/2011 06:44 AM, Fred Drake wrote:
> On Tue, Jun 28, 2011 at 2:33 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> The two terms I've got out of this thread are "callable attributes"
>> (instance/static/class methods, etc) and "data attributes" (everything
>> else). Both seem reasonable to me, creating two largely disjoint sets
>> that together cover all the different kinds of attribute you're likely
>> to encounter.
> 
> But "callable attributes" aren't the same thing as methods; most are methods,
> but not all.  Sometimes, they're data used by the object.  The fact that
> data attributes can be callable is irrelevant.

Isn't it fuzzy / incorrect to refer to any callable attribute as a
method until it have been extracted via the dot operator / getattr, and
therefore bound via descriptor semantics?  In this sense, 'staticmethod'
doesn't create a "method" at all -- it just defeats the default creation
of a method-yielding descriptor.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4JzHcACgkQ+gerLs4ltQ6I5ACZAXlBsZkrzQowKYBmJE4NQb4V
F14AnRtkWByqwpRATan4OOTMgPqwyjxH
=hru9
-----END PGP SIGNATURE-----


From fuzzyman at voidspace.org.uk  Tue Jun 28 14:56:40 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 13:56:40 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <BANLkTimi_=T1AcPn-uMGYgnt90Lwag1OGA@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic>	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>	<87r56en1rt.fsf@benfinney.id.au>	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>	<87mxh2mtq8.fsf@benfinney.id.au>	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>	<4E09B2EF.9030304@voidspace.org.uk>
	<BANLkTimi_=T1AcPn-uMGYgnt90Lwag1OGA@mail.gmail.com>
Message-ID: <4E09CF88.4090307@voidspace.org.uk>

On 28/06/2011 13:31, Nick Coghlan wrote:
> On Tue, Jun 28, 2011 at 8:54 PM, Michael Foord
> <fuzzyman at voidspace.org.uk>  wrote:
>> The problem with "data attributes" is that it doesn't mean *anything*, which
>> I suppose is useful for invented terminology, but it means it doesn't convey
>> anything precise to those who haven't heard the term before. If it becomes
>> widely used then that changes I guess. I'd still normally just use
>> "attributes" though...
> In this context, precision is an illusion. There is *no* precise
> terminology,
"instance members" is precise, (whether or not an attribute is fetched 
"from the instance" is one thing that is clear in the python data model 
- there are just lots of other ways of fetching attributes too). 
"instance members" just doesn't cover all the cases where you may want 
to group attributes together though.

However, providing "__getattr__" and "__getattribute__" is not the 
common case and even properties are *usually* backed by a real instance 
attribute. So "instance attributes" as opposed to other types of 
attributes is *often* a useful distinction.

I don't think "data attributes" is clear or precise. Is a property a 
data attribute (well it depends how it is implemented and what it does), 
is a staticmethod a data attribute (no - but then Tres' question - it 
isn't a normal method either so if you define data attribute to mean 
"all non method attributes" then its potentially an open question).

All the best,

Michael Foord

> not only because Python blurs the boundaries by design in
> many respects (callable or not, method or not, attribute or not, etc),
> but also because *people* use the same words to mean slightly
> different things.
>
> The best we can hope for is to encourage the right way of thinking
> about the situation, and in that regard you have the complete set of
> attributes accessible via an object (i.e. via __getattribute__), some
> of which are callables (and may or may not act like instance methods)
> and the remainder of which are data attributes (some of which may
> incidentally be callable, even if they aren't used that way). How a
> *particular* attribute is classified is not an inherent property of
> the attribute, but also an artifact of the way it is used by the
> application.
>
> Cheers,
> Nick.
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From steve at pearwood.info  Tue Jun 28 15:20:34 2011
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 28 Jun 2011 23:20:34 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09C314.3090809@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>
	<87r56en1rt.fsf@benfinney.id.au>	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>	<87mxh2mtq8.fsf@benfinney.id.au>	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>	<4E09B2EF.9030304@voidspace.org.uk>	<20110628115136.248822505A3@webabinitio.net>
	<4E09C314.3090809@voidspace.org.uk>
Message-ID: <4E09D522.7000009@pearwood.info>

Michael Foord wrote:

> What do you mean by "instances can have methods as instance attributes"? 
> Once you attach a bound method directly to an instance it becomes a 
> slightly different beast I think. (On top of which that is pretty rare 
> behaviour.)

 >>> class C:
...     def method(self, x):
...             return x+1
...
 >>> c = C()
 >>> c.method = types.MethodType(lambda self, x: x+101, c)
 >>> c.method(1)
102

I don't know how rare it is, but it's a useful trick for customising the 
behaviour of instances.


As I see it, there are three dichotomies we sometimes need to make:


(1) Instance attributes vs class (shared) attributes.

Broadly speaking, whether the attribute is in instance.__dict__ or 
type(instance).__dict__.

(2) Computed vs non-computed attributes.

Attributes which are computed by __getattr__ or via the descriptor 
protocol (which includes properties) are all computed attributes; 
everything else is non-computed.

(3) Method attributes (methods) vs non-method/data attributes.

Broadly speaking, methods are callable, non-method (data) attributes are 
not.


The three are orthogonal: e.g. a staticmethod is a method by virtue of 
being callable, computed by virtue of being generated by a descriptor, 
and a class attribute by virtue of existing in the type __dict__ rather 
than the instance __dict__.

Strictly speaking, (3) is not truly a dichotomy, since functions and 
methods are first class-objects in Python. E.g. one may store a function 
as an attribute with the intention of using it as data rather than as a 
method. But that's a moderately obscure corner case, and in my opinion 
it's not worth obscuring the practical distinction between "methods are 
things you call, data are not" for the sake of it. Leave the 
functions-as-data case for a footnote.



-- 
Steven


From fuzzyman at voidspace.org.uk  Tue Jun 28 15:25:13 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 14:25:13 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09CF88.4090307@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic>	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>	<87r56en1rt.fsf@benfinney.id.au>	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>	<87mxh2mtq8.fsf@benfinney.id.au>	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>	<4E09B2EF.9030304@voidspace.org.uk>	<BANLkTimi_=T1AcPn-uMGYgnt90Lwag1OGA@mail.gmail.com>
	<4E09CF88.4090307@voidspace.org.uk>
Message-ID: <4E09D639.5000403@voidspace.org.uk>

On 28/06/2011 13:56, Michael Foord wrote:
> On 28/06/2011 13:31, Nick Coghlan wrote:
>> On Tue, Jun 28, 2011 at 8:54 PM, Michael Foord
>> <fuzzyman at voidspace.org.uk>  wrote:
>>> The problem with "data attributes" is that it doesn't mean 
>>> *anything*, which
>>> I suppose is useful for invented terminology, but it means it 
>>> doesn't convey
>>> anything precise to those who haven't heard the term before. If it 
>>> becomes
>>> widely used then that changes I guess. I'd still normally just use
>>> "attributes" though...
>> In this context, precision is an illusion. There is *no* precise
>> terminology,
> "instance members" is precise, (whether or not an attribute is fetched 
> "from the instance" is one thing that is clear in the python data 
> model - there are just lots of other ways of fetching attributes too). 
> "instance members" just doesn't cover all the cases where you may want 
> to group attributes together though.
>
> However, providing "__getattr__" and "__getattribute__" is not the 
> common case and even properties are *usually* backed by a real 
> instance attribute. So "instance attributes" as opposed to other types 
> of attributes is *often* a useful distinction.
>
> I don't think "data attributes" is clear or precise. 

Although to be fair "data attributes" mirrors the terminology for 
descriptors where we have "data descriptors" (which have both __set__ 
and __get__) and "non data descriptors" (__get__ only). Unfortunately 
there are also non-non-data decriptors too (only __set__ and / or 
__delete__) so that terminology is at least slightly confusing / 
imprecise [1]... There was a genuine Python "bug" caused by that 
confusion at one point (unfortunately I forget what it was).

All the best,

Michael

[1] From: http://users.rcn.com/python/download/Descriptor.htm

     If an object defines both __get__ and __set__, it is considered a 
data descriptor. Descriptors that only define __get__ are called 
non-data descriptors (they are typically used for methods but other uses 
are possible).


> Is a property a data attribute (well it depends how it is implemented 
> and what it does), is a staticmethod a data attribute (no - but then 
> Tres' question - it isn't a normal method either so if you define data 
> attribute to mean "all non method attributes" then its potentially an 
> open question).
>
> All the best,
>
> Michael Foord
>
>> not only because Python blurs the boundaries by design in
>> many respects (callable or not, method or not, attribute or not, etc),
>> but also because *people* use the same words to mean slightly
>> different things.
>>
>> The best we can hope for is to encourage the right way of thinking
>> about the situation, and in that regard you have the complete set of
>> attributes accessible via an object (i.e. via __getattribute__), some
>> of which are callables (and may or may not act like instance methods)
>> and the remainder of which are data attributes (some of which may
>> incidentally be callable, even if they aren't used that way). How a
>> *particular* attribute is classified is not an inherent property of
>> the attribute, but also an artifact of the way it is used by the
>> application.
>>
>> Cheers,
>> Nick.
>>
>
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From fuzzyman at voidspace.org.uk  Tue Jun 28 15:27:32 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 14:27:32 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09D522.7000009@pearwood.info>
References: <20110626185242.GB2710@mathmagic>	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>	<87r56en1rt.fsf@benfinney.id.au>	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>	<87mxh2mtq8.fsf@benfinney.id.au>	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>	<4E09B2EF.9030304@voidspace.org.uk>	<20110628115136.248822505A3@webabinitio.net>	<4E09C314.3090809@voidspace.org.uk>
	<4E09D522.7000009@pearwood.info>
Message-ID: <4E09D6C4.1050505@voidspace.org.uk>

On 28/06/2011 14:20, Steven D'Aprano wrote:
> Michael Foord wrote:
>
>> What do you mean by "instances can have methods as instance 
>> attributes"? Once you attach a bound method directly to an instance 
>> it becomes a slightly different beast I think. (On top of which that 
>> is pretty rare behaviour.)
>
> >>> class C:
> ...     def method(self, x):
> ...             return x+1
> ...
> >>> c = C()
> >>> c.method = types.MethodType(lambda self, x: x+101, c)
> >>> c.method(1)
> 102
>
> I don't know how rare it is, but it's a useful trick for customising 
> the behaviour of instances.
>
Right - that method is an instance attribute.

>
> As I see it, there are three dichotomies we sometimes need to make:
>
>
> (1) Instance attributes vs class (shared) attributes.
>
> Broadly speaking, whether the attribute is in instance.__dict__ or 
> type(instance).__dict__.
>
> (2) Computed vs non-computed attributes.
>
> Attributes which are computed by __getattr__ or via the descriptor 
> protocol (which includes properties) are all computed attributes; 
> everything else is non-computed.
Technically also via __getattribute__ when overridden.

>
> (3) Method attributes (methods) vs non-method/data attributes.
>
> Broadly speaking, methods are callable, non-method (data) attributes 
> are not.
>
>
> The three are orthogonal: e.g. a staticmethod is a method by virtue of 
> being callable, computed by virtue of being generated by a descriptor, 
> and a class attribute by virtue of existing in the type __dict__ 
> rather than the instance __dict__.
>
> Strictly speaking, (3) is not truly a dichotomy, since functions and 
> methods are first class-objects in Python. E.g. one may store a 
> function as an attribute with the intention of using it as data rather 
> than as a method. But that's a moderately obscure corner case, and in 
> my opinion it's not worth obscuring the practical distinction between 
> "methods are things you call, data are not" for the sake of it. Leave 
> the functions-as-data case for a footnote.
>
Yep, useful summary.

Michael


>
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From ncoghlan at gmail.com  Tue Jun 28 15:28:28 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2011 23:28:28 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09CF88.4090307@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
	<BANLkTimi_=T1AcPn-uMGYgnt90Lwag1OGA@mail.gmail.com>
	<4E09CF88.4090307@voidspace.org.uk>
Message-ID: <BANLkTikOKo-nAQ-gtUJ75pUxkaPTVCQo1A@mail.gmail.com>

On Tue, Jun 28, 2011 at 10:56 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> I don't think "data attributes" is clear or precise. Is a property a data
> attribute (well it depends how it is implemented and what it does), is a
> staticmethod a data attribute (no - but then Tres' question - it isn't a
> normal method either so if you define data attribute to mean "all non method
> attributes" then its potentially an open question).

"callable attributes" and "data attributes" are purely about the API
exposed by the object in question. If we're going for object model
neutral terminology, that's the only view that makes sense. They're
descriptions about how attributes are intended to be *used* that are
completely silent on the question of how they're *implemented*. So
staticmethod would fall into the first group, while property would
fall into the latter.

>From an implementation point of view, you carve up the world
differently, so it makes sense to worry about class attributes,
instance attributes, dynamic attributes, etc. (the class vs instance
distinction can also matter to some degree from the usage point of
view, since it affects the scope of any mutable attributes, and the
static vs dynamic distinction can also matter, especially for
introspection purposes).

This goes back to the original point about all of this being highly
context dependent - how you carve up the set of all attributes is
going to change based on what you're trying to explain (e.g. the
distinction between enforced 'data' descriptors, descriptors that
allow shadowing in the instance dict, class attributes that aren't
descriptors at all, instance attributes and dynamic attributes
retrieved via __getattr__ is another way of dividing them)

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From ncoghlan at gmail.com  Tue Jun 28 15:37:34 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 28 Jun 2011 23:37:34 +1000
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09D6C4.1050505@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
	<20110628115136.248822505A3@webabinitio.net>
	<4E09C314.3090809@voidspace.org.uk>
	<4E09D522.7000009@pearwood.info>
	<4E09D6C4.1050505@voidspace.org.uk>
Message-ID: <BANLkTinP85CZYcCjQGjx0js0GLcGdaZGRw@mail.gmail.com>

On Tue, Jun 28, 2011 at 11:27 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Technically also via __getattribute__ when overridden.

Since object.__getattribute__ is the hook that implements the entire
attribute lookup protocol, *all* attributes are technically retrieved
via __getattribute__ (which is why overriding it correctly can be such
a PITA). That's also the hook type() overrides to make class attribute
lookup differ from ordinary instance lookup.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From fuzzyman at voidspace.org.uk  Tue Jun 28 15:41:14 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 14:41:14 +0100
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <20110624145516.84DA32506EA@webabinitio.net>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
Message-ID: <4E09D9FA.7060308@voidspace.org.uk>

On 24/06/2011 15:55, R. David Murray wrote:
> On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinson<dickinsm at gmail.com>  wrote:
>> EuroPython 2011 Language Summit
>> ===============================
> [...]
>>      Unicode character classes is a particular need).  [Subtopic:  what needs
>>      to be done to get the new regex module into Python?  Should it replace
>>      the existing module?  What about backwards compatibility issues?]
> I'm pretty sure regex has backward compatibility as a goal for just this
> reason (so it can replace the current module).
>
The new regex library has some great improvements:

     http://bugs.python.org/issue2636

It also has users and committed maintainers, so I hope we can bring it 
into 3.3. It wasn't easy to tell from skimming the change notes that 
Unicode character classes are amongst the new features. Is that the case?

Michael


>> PEP 0380: Syntax for delegating to a subgenerator.
>>
>> PEP 3150: Statement local namespaces.
>>
>> PEP 3152: Cofunctions.
>>
>> For all three of the above PEPs, there was some feeling that additional syntax
>> changes to the language are unnecessary and make it harder to learn; where
>> possible, we should prefer using 3rd partly libraries.
> I disagree with this with respect to 380.  Haven't looked at the others.
>
>> Python 2.7
>> ----------
>>
>> How long will this be maintained?
>> Original decision was 5 years.
> As I recall, the actual decision was "at *least* 5 years".  It's only
> been one so far...was the discussion that five years was too short or
> too long?
>
> As the code bases continue to drift apart, and we fix the fixable bugs,
> the number of patches going in to 2.7 will decrease over time, so I
> don't think the burden of continuing to support it will be too heavy.
> Currently about half of the non-feature-request issues (ie: bugs) in
> the tracker are tagged 2.7.  I expect to see that percentage continue
> to decrease over time.
>
> --
> R. David Murray           http://www.bitdance.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From victor.stinner at haypocalc.com  Tue Jun 28 15:43:05 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 28 Jun 2011 15:43:05 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in Python
	3.3?
Message-ID: <1309268585.21006.22.camel@marge>

In Python 2, open() opens the file in binary mode (e.g. file.readline()
returns a byte string). codecs.open() opens the file in binary mode by
default, you have to specify an encoding name to open it in text mode.

In Python 3, open() opens the file in text mode by default. (It only
opens the binary mode if the file mode contains "b".) The problem is
that open() uses the locale encoding if the encoding is not specified,
which is the case *by default*. The locale encoding can be:

 - UTF-8 on Mac OS X, most Linux distributions
 - ISO-8859-1 os some FreeBSD systems
 - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
Western Europe, cp952 in Japan, ...
 - ASCII if the locale is manually set to an empty string or to "C", or
if the environment is empty, or by default on some systems
 - something different depending on the system and user configuration...

If you develop under Mac OS X or Linux, you may have surprises when you
run your program on Windows on the first non-ASCII character. You may
not detect the problem if you only write text in english... until
someone writes the first letter with a diacritic.



As discussed before on this list, I propose to set the default encoding
of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
open() is called without an explicit encoding and if the locale encoding
is not UTF-8. Using the warning, you will quickly notice the potential
problem (using Python 3.2.2 and -Werror) on Windows or by using a
different locale encoding (.e.g using LANG="C").

I expect a lot of warnings from the Python standard library, and as many
in third party modules and applications. So do you think that it is too
late to change that in Python 3.3? One argument for changing it directly
in Python 3.3 is that most users will not notice the change because
their locale encoding is already UTF-8.

An alternative is to:
 - Python 3.2: use the locale encoding but emit a warning if the locale
encoding is not UTF-8
 - Python 3.3: use UTF-8 and emit a warning if the locale encoding is
not UTF-8... or maybe always emit a warning?
 - Python 3.3: use UTF-8 (but don't emit warnings anymore)

I don't think that Windows developer even know that they are writing
files into the ANSI code page. MSDN documentation of
WideCharToMultiByte() warns developer that the ANSI code page is not
portable, even accross Windows computers:

"The ANSI code pages can be different on different computers, or can be
changed for a single computer, leading to data corruption. For the most
consistent results, applications should use Unicode, such as UTF-8 or
UTF-16, instead of a specific code page, unless legacy standards or data
formats prevent the use of Unicode. If using Unicode is not possible,
applications should tag the data stream with the appropriate encoding
name when protocols allow it. HTML and XML files allow tagging, but text
files do not."

It will always be possible to use ANSI code page using
encoding="mbcs" (only work on Windows), or an explicit code page number
(e.g. encoding="cp2152").

--

The two other (rejetected?) options to improve open() are:

- raise an error if the encoding argument is not set: will break most
programs
- emit a warning if the encoding argument is not set

--

Should I convert this email into a PEP, or is it not required?

Victor


From mal at egenix.com  Tue Jun 28 16:02:52 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 28 Jun 2011 16:02:52 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <1309268585.21006.22.camel@marge>
References: <1309268585.21006.22.camel@marge>
Message-ID: <4E09DF0C.90200@egenix.com>

Victor Stinner wrote:
> In Python 2, open() opens the file in binary mode (e.g. file.readline()
> returns a byte string). codecs.open() opens the file in binary mode by
> default, you have to specify an encoding name to open it in text mode.
> 
> In Python 3, open() opens the file in text mode by default. (It only
> opens the binary mode if the file mode contains "b".) The problem is
> that open() uses the locale encoding if the encoding is not specified,
> which is the case *by default*. The locale encoding can be:
> 
>  - UTF-8 on Mac OS X, most Linux distributions
>  - ISO-8859-1 os some FreeBSD systems
>  - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
> Western Europe, cp952 in Japan, ...
>  - ASCII if the locale is manually set to an empty string or to "C", or
> if the environment is empty, or by default on some systems
>  - something different depending on the system and user configuration...
> 
> If you develop under Mac OS X or Linux, you may have surprises when you
> run your program on Windows on the first non-ASCII character. You may
> not detect the problem if you only write text in english... until
> someone writes the first letter with a diacritic.

How about a more radical change: have open() in Py3 default to
opening the file in binary mode, if no encoding is given (even
if the mode doesn't include 'b') ?

That'll make it compatible to the Py2 world again and avoid
all the encoding guessing.

Making such default encodings depend on the locale has already
failed to work when we first introduced a default encoding in
Py2, so I don't understand why we are repeating the same
mistake again in Py3 (only in a different area).

Note that in Py2, Unix applications often leave out the 'b'
mode, since there's no difference between using it or not.
Only on Windows, you'll see a difference.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 28 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From victor.stinner at haypocalc.com  Tue Jun 28 16:06:58 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 28 Jun 2011 16:06:58 +0200
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <4E09D9FA.7060308@voidspace.org.uk>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<4E09D9FA.7060308@voidspace.org.uk>
Message-ID: <1309270018.21006.26.camel@marge>

Le mardi 28 juin 2011 ? 14:41 +0100, Michael Foord a ?crit :
> On 24/06/2011 15:55, R. David Murray wrote:
> > On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinson<dickinsm at gmail.com>  wrote:
> >> EuroPython 2011 Language Summit
> >> ===============================
> > [...]
> >>      Unicode character classes is a particular need).  [Subtopic:  what needs
> >>      to be done to get the new regex module into Python?  Should it replace
> >>      the existing module?  What about backwards compatibility issues?]
> > I'm pretty sure regex has backward compatibility as a goal for just this
> > reason (so it can replace the current module).
> >
> The new regex library has some great improvements:
> 
>      http://bugs.python.org/issue2636

This issue is open since April 2008, has also the longest list of
attached files, and has a very long history. What is the status of the
issue? I see that there is now a third party project on:

   http://code.google.com/p/mrab-regex-hg/

--

There is also the re2 library from Google and especially this project:

   http://pypi.python.org/pypi/re2/

"pyre2 is a Python extension that wraps Google's RE2 regular expression
library.

This version of pyre2 is similar to the one you'd find at facebook's
github repository except that the stated goal of *this version is to be
a drop-in replacement for the re module*.)"

Victor


From solipsis at pitrou.net  Tue Jun 28 16:06:17 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 28 Jun 2011 16:06:17 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
References: <1309268585.21006.22.camel@marge>
Message-ID: <20110628160617.56d11422@pitrou.net>

On Tue, 28 Jun 2011 15:43:05 +0200
Victor Stinner <victor.stinner at haypocalc.com> wrote:
>  - ISO-8859-1 os some FreeBSD systems
>  - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
> Western Europe, cp952 in Japan, ...
>  - ASCII if the locale is manually set to an empty string or to "C", or
> if the environment is empty, or by default on some systems
>  - something different depending on the system and user configuration...

Why would utf-8 be the right thing in these cases?

Regards

Antoine.



From tjreedy at udel.edu  Tue Jun 28 16:24:58 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Jun 2011 10:24:58 -0400
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <1309268585.21006.22.camel@marge>
References: <1309268585.21006.22.camel@marge>
Message-ID: <iuco7n$k7r$1@dough.gmane.org>

On 6/28/2011 9:43 AM, Victor Stinner wrote:
> In Python 2, open() opens the file in binary mode (e.g. file.readline()
> returns a byte string). codecs.open() opens the file in binary mode by
> default, you have to specify an encoding name to open it in text mode.
>
> In Python 3, open() opens the file in text mode by default. (It only
> opens the binary mode if the file mode contains "b".) The problem is
> that open() uses the locale encoding if the encoding is not specified,
> which is the case *by default*. The locale encoding can be:
>
>   - UTF-8 on Mac OS X, most Linux distributions
>   - ISO-8859-1 os some FreeBSD systems
>   - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
> Western Europe, cp952 in Japan, ...
>   - ASCII if the locale is manually set to an empty string or to "C", or
> if the environment is empty, or by default on some systems
>   - something different depending on the system and user configuration...
>
> If you develop under Mac OS X or Linux, you may have surprises when you
> run your program on Windows on the first non-ASCII character. You may
> not detect the problem if you only write text in english... until
> someone writes the first letter with a diacritic.
>
>
>
> As discussed before on this list, I propose to set the default encoding
> of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
> open() is called without an explicit encoding and if the locale encoding
> is not UTF-8. Using the warning, you will quickly notice the potential
> problem (using Python 3.2.2 and -Werror) on Windows or by using a
> different locale encoding (.e.g using LANG="C").
>
> I expect a lot of warnings from the Python standard library, and as many
> in third party modules and applications. So do you think that it is too
> late to change that in Python 3.3? One argument for changing it directly
> in Python 3.3 is that most users will not notice the change because
> their locale encoding is already UTF-8.
>
> An alternative is to:
>   - Python 3.2: use the locale encoding but emit a warning if the locale
> encoding is not UTF-8
>   - Python 3.3: use UTF-8 and emit a warning if the locale encoding is
> not UTF-8... or maybe always emit a warning?
>   - Python 3.3: use UTF-8 (but don't emit warnings anymore)
>
> I don't think that Windows developer even know that they are writing
> files into the ANSI code page. MSDN documentation of
> WideCharToMultiByte() warns developer that the ANSI code page is not
> portable, even accross Windows computers:
>
> "The ANSI code pages can be different on different computers, or can be
> changed for a single computer, leading to data corruption. For the most
> consistent results, applications should use Unicode, such as UTF-8 or
> UTF-16, instead of a specific code page, unless legacy standards or data
> formats prevent the use of Unicode. If using Unicode is not possible,
> applications should tag the data stream with the appropriate encoding
> name when protocols allow it. HTML and XML files allow tagging, but text
> files do not."
>
> It will always be possible to use ANSI code page using
> encoding="mbcs" (only work on Windows), or an explicit code page number
> (e.g. encoding="cp2152").
>
> --
>
> The two other (rejetected?) options to improve open() are:
>
> - raise an error if the encoding argument is not set: will break most
> programs
> - emit a warning if the encoding argument is not set
>
> --
>
> Should I convert this email into a PEP, or is it not required?

I think a PEP is needed.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Tue Jun 28 16:36:23 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Jun 2011 10:36:23 -0400
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <4E09DF0C.90200@egenix.com>
References: <1309268585.21006.22.camel@marge> <4E09DF0C.90200@egenix.com>
Message-ID: <iucot6$pma$1@dough.gmane.org>

On 6/28/2011 10:02 AM, M.-A. Lemburg wrote:

> How about a more radical change: have open() in Py3 default to
> opening the file in binary mode, if no encoding is given (even
> if the mode doesn't include 'b') ?
>
> That'll make it compatible to the Py2 world again

I disagree. I believe
S = open('myfile.txt').read()
now return a text string in both Py2 and Py3 and a subsequent
'abc' in S
works in both.

 > and avoid all the encoding guessing.

> Making such default encodings depend on the locale has already
> failed to work when we first introduced a default encoding in
> Py2, so I don't understand why we are repeating the same
> mistake again in Py3 (only in a different area).

I do not remember any proposed change during the Py3 design discussions.

> Note that in Py2, Unix applications often leave out the 'b'
> mode, since there's no difference between using it or not.

I believe it makes a difference now as to whether one gets str or bytes.

> Only on Windows, you'll see a difference.

I believe the only difference now on Windows is the decoding used, not 
the return type.

-- 
Terry Jan Reedy


From benjamin at python.org  Tue Jun 28 16:40:22 2011
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 28 Jun 2011 09:40:22 -0500
Subject: [Python-Dev] cpython: remove duplicate -I
In-Reply-To: <nad-67ECB0.01171628062011@news.gmane.org>
References: <E1QYQFe-0006Ir-1o@dinsdale.python.org>
	<nad-67ECB0.01171628062011@news.gmane.org>
Message-ID: <BANLkTik3rEerDjEKS644BFqBd-gDR0KBOA@mail.gmail.com>

2011/6/28 Ned Deily <nad at acm.org>:
> In article <E1QYQFe-0006Ir-1o at dinsdale.python.org>,
> ?benjamin.peterson <python-checkins at python.org> wrote:
>
>> http://hg.python.org/cpython/rev/0d0ca6f95d9d
>> changeset: ? 70879:0d0ca6f95d9d
>> user: ? ? ? ?Benjamin Peterson <benjamin at python.org>
>> date: ? ? ? ?Sun Jun 19 17:17:30 2011 -0500
>> summary:
>> ? remove duplicate -I
>>
>> files:
>> ? Makefile.pre.in | ?2 +-
>> ? 1 files changed, 1 insertions(+), 1 deletions(-)
>>
>>
>> diff --git a/Makefile.pre.in b/Makefile.pre.in
>> --- a/Makefile.pre.in
>> +++ b/Makefile.pre.in
>> @@ -73,7 +73,7 @@
>> ?# Both CPPFLAGS and LDFLAGS need to contain the shell's value for setup.py
>> ?to
>> ?# be able to build extension modules using the directories specified in the
>> ?# environment variables
>> -PY_CPPFLAGS= -I. -IInclude -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS)
>> $(CPPFLAGS)
>> +PY_CPPFLAGS= -I. -IInclude $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)
>> ?PY_LDFLAGS= ?$(CONFIGURE_LDFLAGS) $(LDFLAGS)
>> ?NO_AS_NEEDED= ? ? ? ?@NO_AS_NEEDED@
>> ?LDLAST= ? ? ? ? ? ? ?@LDLAST@
>
> I think someone has already mentioned that this change breaks builds
> with a source directory outside the build directory and that breaks my
> OS X installer builds. ? Please revert or make the following change
> which works for me (but not tested elsewhere):
>
> -PY_CPPFLAGS= ? -I. -IInclude $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)
> +PY_CPPFLAGS= ? -I. -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)

Committed. Thanks for the review.


-- 
Regards,
Benjamin

From tjreedy at udel.edu  Tue Jun 28 16:41:38 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Jun 2011 10:41:38 -0400
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <20110628160617.56d11422@pitrou.net>
References: <1309268585.21006.22.camel@marge>
	<20110628160617.56d11422@pitrou.net>
Message-ID: <iucp72$rn2$1@dough.gmane.org>

On 6/28/2011 10:06 AM, Antoine Pitrou wrote:
> On Tue, 28 Jun 2011 15:43:05 +0200
> Victor Stinner<victor.stinner at haypocalc.com>  wrote:
>>   - ISO-8859-1 os some FreeBSD systems
>>   - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
>> Western Europe, cp952 in Japan, ...
>>   - ASCII if the locale is manually set to an empty string or to "C", or
>> if the environment is empty, or by default on some systems
>>   - something different depending on the system and user configuration...
>
> Why would utf-8 be the right thing in these cases?

Because utf-8 is the only way to write out any Python 3 text.
By default, writing and reading an str object should work on all Python 
installations.
And because other apps are (increasingly) using it for exactly the same 
reason.


-- 
Terry Jan Reedy


From p.f.moore at gmail.com  Tue Jun 28 16:46:12 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 28 Jun 2011 15:46:12 +0100
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <1309268585.21006.22.camel@marge>
References: <1309268585.21006.22.camel@marge>
Message-ID: <BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>

On 28 June 2011 14:43, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> As discussed before on this list, I propose to set the default encoding
> of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
> open() is called without an explicit encoding and if the locale encoding
> is not UTF-8. Using the warning, you will quickly notice the potential
> problem (using Python 3.2.2 and -Werror) on Windows or by using a
> different locale encoding (.e.g using LANG="C").

-1. This will make things harder for simple scripts which are not
intended to be cross-platform.

I use Windows, and come from the UK, so 99% of my text files are
ASCII. So the majority of my code will be unaffected. But in the
occasional situation where I use a ? sign, I'll get encoding errors,
where currently things will "just work". And the failures will be data
dependent, and hence intermittent (the worst type of problem). I'll
write a quick script, use it once and it'll be fine, then use it later
on some different data and get an error. :-(

I appreciate that the point here is to make sure that people think a
bit more carefully about encoding issues. But doing so by making
Python less friendly for casual, adhoc script use, seems to me to be a
mistake.

> I don't think that Windows developer even know that they are writing
> files into the ANSI code page. MSDN documentation of
> WideCharToMultiByte() warns developer that the ANSI code page is not
> portable, even accross Windows computers:

Probably true. But for many uses they also don't care. If you're
writing something solely for a one-off job on your own PC, the ANSI
code page is fine, and provides interoperability with other programs
on your PC, which is really what you care about. (UTF-8 without BOM
displays incorrectly in Vim, wordpad, and powershell get-content. MBCS
works fine in all of these. It also displays incorrectly in CMD type,
but in a less familiar form than the incorrect display mbcs produces,
for what that's worth...)

> It will always be possible to use ANSI code page using
> encoding="mbcs" (only work on Windows), or an explicit code page number
> (e.g. encoding="cp2152").

So, in effect, you propose making the default favour writing
multiplatform portable code at the expense of quick and dirty scripts?
My personal view is that this is the wrong choice ("practicality beats
purity") but I guess it's ultimately a question of Python's design
philosophy.

> The two other (rejetected?) options to improve open() are:
>
> - raise an error if the encoding argument is not set: will break most
> programs
> - emit a warning if the encoding argument is not set

IMHO, you missed another option - open() does not need improving, the
current behaviour is better than any of the 3 options noted.

Paul.

From fuzzyman at voidspace.org.uk  Tue Jun 28 16:48:09 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 15:48:09 +0100
Subject: [Python-Dev] open(): set the default encoding to 'utf-8'
 in	Python 3.3?
In-Reply-To: <iucot6$pma$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge> <4E09DF0C.90200@egenix.com>
	<iucot6$pma$1@dough.gmane.org>
Message-ID: <4E09E9A9.5020702@voidspace.org.uk>

On 28/06/2011 15:36, Terry Reedy wrote:
> On 6/28/2011 10:02 AM, M.-A. Lemburg wrote:
>
>> How about a more radical change: have open() in Py3 default to
>> opening the file in binary mode, if no encoding is given (even
>> if the mode doesn't include 'b') ?
>>
>> That'll make it compatible to the Py2 world again
>
> I disagree. I believe
> S = open('myfile.txt').read()
> now return a text string in both Py2 and Py3 and a subsequent
> 'abc' in S
> works in both.

Nope, it returns a bytestring in Python 2. Mistakenly treating 
bytestrings as text is one of the things we aimed to correct in the 
transition to Python 3.

Michael

>
> > and avoid all the encoding guessing.
>
>> Making such default encodings depend on the locale has already
>> failed to work when we first introduced a default encoding in
>> Py2, so I don't understand why we are repeating the same
>> mistake again in Py3 (only in a different area).
>
> I do not remember any proposed change during the Py3 design discussions.
>
>> Note that in Py2, Unix applications often leave out the 'b'
>> mode, since there's no difference between using it or not.
>
> I believe it makes a difference now as to whether one gets str or bytes.
>
>> Only on Windows, you'll see a difference.
>
> I believe the only difference now on Windows is the decoding used, not 
> the return type.
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From solipsis at pitrou.net  Tue Jun 28 17:02:52 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 28 Jun 2011 17:02:52 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
References: <1309268585.21006.22.camel@marge>
	<20110628160617.56d11422@pitrou.net> <iucp72$rn2$1@dough.gmane.org>
Message-ID: <20110628170252.7260dc65@pitrou.net>

On Tue, 28 Jun 2011 10:41:38 -0400
Terry Reedy <tjreedy at udel.edu> wrote:
> On 6/28/2011 10:06 AM, Antoine Pitrou wrote:
> > On Tue, 28 Jun 2011 15:43:05 +0200
> > Victor Stinner<victor.stinner at haypocalc.com>  wrote:
> >>   - ISO-8859-1 os some FreeBSD systems
> >>   - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
> >> Western Europe, cp952 in Japan, ...
> >>   - ASCII if the locale is manually set to an empty string or to "C", or
> >> if the environment is empty, or by default on some systems
> >>   - something different depending on the system and user configuration...
> >
> > Why would utf-8 be the right thing in these cases?
> 
> Because utf-8 is the only way to write out any Python 3 text.

Er, no, you also have utf-16, utf-32, utf-7 (and possibly others,
including home-baked encodings).

> By default, writing and reading an str object should work on all Python 
> installations.

But that's only half of the problem. If the text is supposed to be read
or processed by some other program, then writing it in some encoding
that the other program doesn't expect doesn't really help. That's why
we use the locale encoding: because it's a good guess as to what the
system (and its users) expects text to be encoded in.

Regards

Antoine.



From sdaoden at googlemail.com  Tue Jun 28 17:06:41 2011
From: sdaoden at googlemail.com (Steffen Daode Nurpmeso)
Date: Tue, 28 Jun 2011 17:06:41 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
Message-ID: <20110628150641.GA29535@sherwood.local>

@ Paul Moore <p.f.moore at gmail.com> wrote (2011-06-28 16:46+0200):
> UTF-8 without BOM displays incorrectly in vim(1)

Stop right now (you're oh so wrong)!  :-)

(By the way: UTF-8 and BOM?
Interesting things i learn on this list.
And i hope in ten years we can laugh about this -> UTF-8
transition all over the place, 'cause it's simply working.)

--
Ciao, Steffen
sdaoden(*)(gmail.com)
() ascii ribbon campaign - against html e-mail
/\ www.asciiribbon.org - against proprietary attachments

From tjreedy at udel.edu  Tue Jun 28 17:23:33 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Jun 2011 11:23:33 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <20110628115136.248822505A3@webabinitio.net>
References: <20110626185242.GB2710@mathmagic>
	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>
	<87r56en1rt.fsf@benfinney.id.au>	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>	<87mxh2mtq8.fsf@benfinney.id.au>	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>	<4E09B2EF.9030304@voidspace.org.uk>
	<20110628115136.248822505A3@webabinitio.net>
Message-ID: <iucrll$cfi$1@dough.gmane.org>

On 6/28/2011 7:51 AM, R. David Murray wrote:

> Also, instances can have methods as instance attributes.

Functions that are instance attributes do not act like methods 
(instance.func() does not automagically turn instance in the first arg 
of func) and have never, to my knowledge, been called methods. In Python 
2, they are not wrapped as methods whereas functions attached to classes 
are.

So-called 'staticmethods' are not really methods either, but are class 
function attributes that are just functions and not treated as methods. 
The decorator that negates normal method treatment could/should have 
been called 'non_method'.

Using 'function' is its generic 'callable' sense ...

Method: a class function attribute that in its intended and normal use 
automagically turns the object it is called on into its first arg. 
'Method' is a useful and needed subcategory of class attribute precisely 
because of this behavior.

Instance method: a class function attribute that is (normally) called on 
instances of the class or subclasses.

Class method: a class function attribute that is (normally) called on 
the class or subclasses.

Bound method: a method that has already has the first-arg object bundled 
with it, so that it can be used as a normal (partial or curried) function.

Except for 'classmethod', which was added later, these have been the 
meanings as I understood them since at least Py 1.4, 15 years ago, and 
they are exactly what one needs to know to use Python. Any object can be 
an attribute. However, function attributes of classes normally get 
special 'method' treatment which alters the meaning of syntax.

-- 
Terry Jan Reedy


From ericsnowcurrently at gmail.com  Tue Jun 28 17:35:01 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Tue, 28 Jun 2011 09:35:01 -0600
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09BC8C.7090807@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
	<BANLkTimC6=OAWHLK7_XU+teUCQmugA=zgw@mail.gmail.com>
	<4E09BC8C.7090807@voidspace.org.uk>
Message-ID: <BANLkTi=S2-1e2wjFL6Y=8KYEJ7DGaqxTBQ@mail.gmail.com>

On Tue, Jun 28, 2011 at 5:35 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
>
> Well, they're *usually* implemented as methods and backed by a real instance
> attribute. Usually (but not always) it makes more sense (IME) to group them
> with methods. The fact that they're *accessed* as an attribute is the
> uninteresting detail.
>
> __getattr__ and __getattribute__ are interesting - they allow you to use
> attribute access syntax for things that *aren't* attributes.
>
> I appreciate the fact that the Python data-model means methods are just
> object attributes, but they're not *instance attributes* and sometimes you
> need to make a distinction. (And yes it is helpful if the standard
> terminology leads people into a better understanding of the Python data
> model, but that still doesn't change the need on occasion for terminology
> that doesn't need to be explained whenever it is used.)
>

While the distinction between class-specific attributes and
instance-specific is important, I don't think merging it with the
method/data distinction is as helpful.  The valuable information here
is the expectation of how the attributes are used.  In my mind, that
boils down to data (holds static or dymanic state) and methods (does
something on the class or instance).

-eric

> Given that how even methods are to be described depends on the context (if
> you're fetching bound methods as objects then it makes perfect sense to just
> talk about them as attributes) it doesn't seem an area amenable to
> one-size-fits-all terminology.
>
>>> The problem with "data attributes" is that it doesn't mean *anything*,
>>> which
>>> I suppose is useful for invented terminology, but it means it doesn't
>>> convey
>>> anything precise to those who haven't heard the term before. If it
>>> becomes
>>> widely used then that changes I guess. I'd still normally just use
>>> "attributes" though...
>>
>> I'd read "data attributes" the same as "non-method attributes". ?For
>> readers,
>> calling them "attributes" is typically sufficient. ?It's rare to need to
>> distinguish them from methods.
>
> Yeah, this is all a grand bikeshed. I'm not sure I would understand "data
> attributes" unless it was clear from the context. I would wonder what
> qualifies something as "data".
>
> It is an interesting question what terminology we should use in the
> documentation if we need to distinguish them, but I think that is still
> wandering away from the original question that was posed.
>
> All the best,
>
> Michael
>
>>
>> ? -Fred
>>
>
>
> --
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing http://www.sqlite.org/different.html
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>

From fuzzyman at voidspace.org.uk  Tue Jun 28 17:35:38 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 16:35:38 +0100
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <iucrll$cfi$1@dough.gmane.org>
References: <20110626185242.GB2710@mathmagic>	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>	<87r56en1rt.fsf@benfinney.id.au>	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>	<87mxh2mtq8.fsf@benfinney.id.au>	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>	<4E09B2EF.9030304@voidspace.org.uk>	<20110628115136.248822505A3@webabinitio.net>
	<iucrll$cfi$1@dough.gmane.org>
Message-ID: <4E09F4CA.3060002@voidspace.org.uk>

On 28/06/2011 16:23, Terry Reedy wrote:
> On 6/28/2011 7:51 AM, R. David Murray wrote:
>
>> Also, instances can have methods as instance attributes.
>
> Functions that are instance attributes do not act like methods 
> (instance.func() does not automagically turn instance in the first arg 
> of func) and have never, to my knowledge, been called methods. In 
> Python 2, they are not wrapped as methods whereas functions attached 
> to classes are.
>
> So-called 'staticmethods' are not really methods either, but are class 
> function attributes that are just functions and not treated as 
> methods. The decorator that negates normal method treatment 
> could/should have been called 'non_method'.
>
> Using 'function' is its generic 'callable' sense ...
>
> Method: a class function attribute that in its intended and normal use 
> automagically turns the object it is called on into its first arg. 
> 'Method' is a useful and needed subcategory of class attribute 
> precisely because of this behavior.
>
> Instance method: a class function attribute that is (normally) called 
> on instances of the class or subclasses.
>

So what is the difference between "Instance method" and "Method" above? 
Is it just that "Method" is broader and includes class methods and bound 
methods?

If anyone said "instance method" to me I would assume they meant bound 
method. (A normal method fetched from an instance.)

All the best,

Michael

> Class method: a class function attribute that is (normally) called on 
> the class or subclasses.
>
> Bound method: a method that has already has the first-arg object 
> bundled with it, so that it can be used as a normal (partial or 
> curried) function.
>
> Except for 'classmethod', which was added later, these have been the 
> meanings as I understood them since at least Py 1.4, 15 years ago, and 
> they are exactly what one needs to know to use Python. Any object can 
> be an attribute. However, function attributes of classes normally get 
> special 'method' treatment which alters the meaning of syntax.
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From ericsnowcurrently at gmail.com  Tue Jun 28 17:42:46 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Tue, 28 Jun 2011 09:42:46 -0600
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09D522.7000009@pearwood.info>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
	<20110628115136.248822505A3@webabinitio.net>
	<4E09C314.3090809@voidspace.org.uk>
	<4E09D522.7000009@pearwood.info>
Message-ID: <BANLkTi=MDukpB8gCmj0ri6BfSmFYVsnANQ@mail.gmail.com>

On Tue, Jun 28, 2011 at 7:20 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> Michael Foord wrote:
>
>> What do you mean by "instances can have methods as instance attributes"?
>> Once you attach a bound method directly to an instance it becomes a slightly
>> different beast I think. (On top of which that is pretty rare behaviour.)
>
>>>> class C:
> ... ? ? def method(self, x):
> ... ? ? ? ? ? ? return x+1
> ...
>>>> c = C()
>>>> c.method = types.MethodType(lambda self, x: x+101, c)
>>>> c.method(1)
> 102
>
> I don't know how rare it is, but it's a useful trick for customising the
> behaviour of instances.
>
>
> As I see it, there are three dichotomies we sometimes need to make:
>
>
> (1) Instance attributes vs class (shared) attributes.
>
> Broadly speaking, whether the attribute is in instance.__dict__ or
> type(instance).__dict__.
>
> (2) Computed vs non-computed attributes.
>
> Attributes which are computed by __getattr__ or via the descriptor protocol
> (which includes properties) are all computed attributes; everything else is
> non-computed.
>
> (3) Method attributes (methods) vs non-method/data attributes.
>
> Broadly speaking, methods are callable, non-method (data) attributes are
> not.
>

For terminology, is it important that data attributes are [usually]
not callable, or is it that being callable is not relevant to their
use as attributes of the class/instance?  The same for "methods".

We have precedent for where the the terminology represents an
expectation rather than a firm constraint (__isabstractmethod__).

>
> The three are orthogonal: e.g. a staticmethod is a method by virtue of being
> callable, computed by virtue of being generated by a descriptor, and a class
> attribute by virtue of existing in the type __dict__ rather than the
> instance __dict__.
>
> Strictly speaking, (3) is not truly a dichotomy, since functions and methods
> are first class-objects in Python. E.g. one may store a function as an
> attribute with the intention of using it as data rather than as a method.
> But that's a moderately obscure corner case, and in my opinion it's not
> worth obscuring the practical distinction between "methods are things you
> call, data are not" for the sake of it. Leave the functions-as-data case for
> a footnote.
>

+1

The "three dichotomies" is a great way to look at it.

-eric

>
>
> --
> Steven
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>

From p.f.moore at gmail.com  Tue Jun 28 17:44:02 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 28 Jun 2011 16:44:02 +0100
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <20110628150641.GA29535@sherwood.local>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
	<20110628150641.GA29535@sherwood.local>
Message-ID: <BANLkTikGD4-mfi5ABN1=L62wF1w7fXK-8w@mail.gmail.com>

On 28 June 2011 16:06, Steffen Daode Nurpmeso <sdaoden at googlemail.com> wrote:
> @ Paul Moore <p.f.moore at gmail.com> wrote (2011-06-28 16:46+0200):
>> UTF-8 without BOM displays incorrectly in vim(1)
>
> Stop right now (you're oh so wrong)! ?:-)

Sorry. Please add "using the default settings of gvim on Windows". My
context throughout was Windows not Unix. Sorry I didn't make that
clear.

> (By the way: UTF-8 and BOM?

Windows uses it, I believe. My tests specifically used files with no
BOM, just utf8-encoded text. I made this statement to head off people
assuming that UTF8 can be detected in Windows by looking at the first
few bytes.

> Interesting things i learn on this list.

:-)

> And i hope in ten years we can laugh about this -> UTF-8
> transition all over the place, 'cause it's simply working.)

That would be good...

Paul.

From tjreedy at udel.edu  Tue Jun 28 18:12:24 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Jun 2011 12:12:24 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
	in documentation
In-Reply-To: <4E09D522.7000009@pearwood.info>
References: <20110626185242.GB2710@mathmagic>	<iu89n5$eg3$1@dough.gmane.org>	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>	<20110627102428.6fc1955b@msiwind>	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>	<87r56en1rt.fsf@benfinney.id.au>	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>	<87mxh2mtq8.fsf@benfinney.id.au>	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>	<4E09B2EF.9030304@voidspace.org.uk>	<20110628115136.248822505A3@webabinitio.net>	<4E09C314.3090809@voidspace.org.uk>
	<4E09D522.7000009@pearwood.info>
Message-ID: <iucuh6$v4l$1@dough.gmane.org>

On 6/28/2011 9:20 AM, Steven D'Aprano wrote:

>  >>> class C:
> ... def method(self, x):
> ... return x+1
> ...
>  >>> c = C()
>  >>> c.method = types.MethodType(lambda self, x: x+101, c)

types.MethodType creates a bound method, not a method. A bound method is 
a partial or curried function, which is to say, a function.

Herw and below, I am ignoring the fact that the example ignores self. My 
comments would be the same for "lambda self,x: return self.value+x"

>  >>> c.method(1)

The fact that you make the bound function an attribute of the same 
object to which it is bound is irrelevant to how it *acts* when called. 
It only affect how you *access* it. You could bind it to anything else, 
including another instance or a plain name:

d = C()
d.method = types.MethodType(lambda self, x: x+101, c)
d.method(1)

m = types.MethodType(lambda self, x: x+101, c)
m(1)

> 102

> I don't know how rare it is, but it's a useful trick for customising the
> behaviour of instances.
>
>
> As I see it, there are three dichotomies we sometimes need to make:
>
>
> (1) Instance attributes vs class (shared) attributes.
> (2) Computed vs non-computed attributes.
> (3) Method attributes (methods) vs non-method/data attributes.

Nicely put so far...

 > The three are orthogonal:

Non-class instance function attributes are not methods in any useful sense.

 > a staticmethod is a method by virtue of being callable

A 'staticmethod' is a non-method class function attribute. The name is 
unfortunate. 'Static' mean 'does not get the dynamic method treatment'.

> Strictly speaking, (3) is not truly a dichotomy,

I disagree here. A method is usefully defined as a class function 
attribute that gets an automagic first arg when accessed and called 
'normally'. In other words, the dichotomy is needed because being a 
method affects the meaning of syntax.

> since functions and
> methods are first class-objects in Python. E.g. one may store a function
> as an attribute with the intention of using it as data rather than as a
> method.

The fact that one can *also* access a method as a function does not 
negate the effect on syntax.

> "methods are things you call"

Way too broad, even for attributes. Callable things are methods when 
'a.b(c)' causes a to be the first arg of b.

-- 
Terry Jan Reedy


From a.badger at gmail.com  Tue Jun 28 18:33:51 2011
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Tue, 28 Jun 2011 09:33:51 -0700
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
Message-ID: <20110628163351.GK27252@unaka.lan>

On Tue, Jun 28, 2011 at 03:46:12PM +0100, Paul Moore wrote:
> On 28 June 2011 14:43, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> > As discussed before on this list, I propose to set the default encoding
> > of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
> > open() is called without an explicit encoding and if the locale encoding
> > is not UTF-8. Using the warning, you will quickly notice the potential
> > problem (using Python 3.2.2 and -Werror) on Windows or by using a
> > different locale encoding (.e.g using LANG="C").
> 
> -1. This will make things harder for simple scripts which are not
> intended to be cross-platform.
> 
> I use Windows, and come from the UK, so 99% of my text files are
> ASCII. So the majority of my code will be unaffected. But in the
> occasional situation where I use a ? sign, I'll get encoding errors,
> where currently things will "just work". And the failures will be data
> dependent, and hence intermittent (the worst type of problem). I'll
> write a quick script, use it once and it'll be fine, then use it later
> on some different data and get an error. :-(
>
I don't think this change would make things "harder".  It will just move
where the pain occurs.  Right now, the failures are intermittent on A)
computers other than the one that you're using. or B) intermittent when run
under a different user than yourself.  Sys admins where I'm at are
constantly writing ad hoc scripts in python that break because you stick
something in a cron job and the locale settings suddenly become "C" and
therefore the script suddenly only deals with ASCII characters.

I don't know that Victor's proposed solution is the best (I personally would
like it a whole lot more than the current guessing but I never develop on
Windows so I can certainly see that your environment can lead to the
opposite assumption :-) but something should change here.  Issuing a warning
like "open used without explicit encoding may lead to errors" if open() is
used without an explicit encoding would help a little (at least, people who
get errors would then have an inkling that the culprit might be an open()
call).  If I read Victor's previous email correctly, though, he said this
was previously rejected.

Another brainstorming solution would be to use different default encodings on
different platforms.  For instance, for writing files, utf-8 on *nix systems
(including macosX) and utf-16 on windows.  For reading files, check for a utf-16
BOM, if not present, operate as utf-8.  That would seem to address your
issue with detection by vim, etc but I'm not sure about getting "?" in your
input stream.  I don't know where your input is coming from and how Windows
equivalent of locale plays into that.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110628/11ac1081/attachment.pgp>

From tjreedy at udel.edu  Tue Jun 28 18:34:56 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Jun 2011 12:34:56 -0400
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <4E09E9A9.5020702@voidspace.org.uk>
References: <1309268585.21006.22.camel@marge>
	<4E09DF0C.90200@egenix.com>	<iucot6$pma$1@dough.gmane.org>
	<4E09E9A9.5020702@voidspace.org.uk>
Message-ID: <iucvrg$885$1@dough.gmane.org>

On 6/28/2011 10:48 AM, Michael Foord wrote:
> On 28/06/2011 15:36, Terry Reedy wrote:

>> S = open('myfile.txt').read()
>> now return a text string in both Py2 and Py3 and a subsequent
>> 'abc' in S
>> works in both.
>
> Nope, it returns a bytestring in Python 2.

Which, in Py2 is a str() object. In both Pythons, .read() in default 
mode returns an object of type str() and 'abc' is an object of type 
str() and so expressions involving undecorated string literals and input 
just work, but would not work if input defaulted to bytes in Py 3. Sorry 
if I was not clear enough.

-- 
Terry Jan Reedy


From nad at acm.org  Tue Jun 28 18:37:28 2011
From: nad at acm.org (Ned Deily)
Date: Tue, 28 Jun 2011 09:37:28 -0700
Subject: [Python-Dev] svn.python.org confusion
Message-ID: <nad-47439B.09372828062011@news.gmane.org>

Not surprisingly, people are still attempting to browse and/or checkout 
Python source from svn.python.org.  They could be following out-of-date 
instructions from somewhere or might be expecting the svn repos are 
mirroring the hg ones.  This causes wasted time and frustration for 
users and for us following up on issues.

Could some text be added to the svn browser pages to at least indicate 
that the repo is no longer being updated with a link to the 
hg.python.org browser?   I'm not sure what to do about the repos 
themselves if people attempt to do an svn co.  Perhaps that should just 
be disabled totally for python?

-- 
 Ned Deily,
 nad at acm.org


From fuzzyman at voidspace.org.uk  Tue Jun 28 18:45:44 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 17:45:44 +0100
Subject: [Python-Dev] svn.python.org confusion
In-Reply-To: <nad-47439B.09372828062011@news.gmane.org>
References: <nad-47439B.09372828062011@news.gmane.org>
Message-ID: <4E0A0538.5000401@voidspace.org.uk>

On 28/06/2011 17:37, Ned Deily wrote:
> Not surprisingly, people are still attempting to browse and/or checkout
> Python source from svn.python.org.  They could be following out-of-date
> instructions from somewhere or might be expecting the svn repos are
> mirroring the hg ones.  This causes wasted time and frustration for
> users and for us following up on issues.
>
> Could some text be added to the svn browser pages to at least indicate
> that the repo is no longer being updated with a link to the
> hg.python.org browser?   I'm not sure what to do about the repos
> themselves if people attempt to do an svn co.  Perhaps that should just
> be disabled totally for python?
>
For what it's worth we've had a couple of emails about this to 
webmaster at python.org (in particular because the viewc svn browser breaks 
on some latin-1 in source files).

All the best,

Michael Foord

-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From janssen at parc.com  Tue Jun 28 18:49:13 2011
From: janssen at parc.com (Bill Janssen)
Date: Tue, 28 Jun 2011 09:49:13 PDT
Subject: [Python-Dev] Snow Leopard buildbot failing again...
In-Reply-To: <20110628102431.2fd841d4@pitrou.net>
References: <26766.1309194073@parc.com> <20110628102431.2fd841d4@pitrou.net>
Message-ID: <43935.1309279753@parc.com>

Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Mon, 27 Jun 2011 10:01:13 PDT
> Bill Janssen <janssen at parc.com> wrote:
> > I see that parc-snowleopard-1 went down again.  I've done a software
> > update, rebooted, and installed the latest buildslave, 0.8.4.  I can
> > ping dinsdale.python.org successfully from the machine.  However, when I
> > start the buildslave, I get this:
> 
> Well, doesn't the slave work correctly now? Or is there a problem?

It does seem to have connected, but the error message from buildslave is
new.  Let's see if it goes into the zombie state again running 0.8.4.

The Tiger buildbot (an old dual-1GHz G4 running buildbot-slave 0.8.1p1)
is the only one which seems to run reliably :-).

Bill

From fuzzyman at voidspace.org.uk  Tue Jun 28 18:50:29 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 17:50:29 +0100
Subject: [Python-Dev] open(): set the default encoding to 'utf-8'
 in	Python 3.3?
In-Reply-To: <iucvrg$885$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge>	<4E09DF0C.90200@egenix.com>	<iucot6$pma$1@dough.gmane.org>	<4E09E9A9.5020702@voidspace.org.uk>
	<iucvrg$885$1@dough.gmane.org>
Message-ID: <4E0A0655.40207@voidspace.org.uk>

On 28/06/2011 17:34, Terry Reedy wrote:
> On 6/28/2011 10:48 AM, Michael Foord wrote:
>> On 28/06/2011 15:36, Terry Reedy wrote:
>
>>> S = open('myfile.txt').read()
>>> now return a text string in both Py2 and Py3 and a subsequent
>>> 'abc' in S
>>> works in both.
>>
>> Nope, it returns a bytestring in Python 2.
>
> Which, in Py2 is a str() object. 

Yes, but not a "text string". The equivalent of the Python 2 str in 
Python 3 is bytes. Irrelevant discussion anyway.

> In both Pythons, .read() in default mode returns an object of type 
> str() and 'abc' is an object of type str() and so expressions 
> involving undecorated string literals and input just work, but would 
> not work if input defaulted to bytes in Py 3. Sorry if I was not clear 
> enough.
>
Well, I think you're both right. Both semantics break some assumption or 
other.

All the best,

Michael

-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From janssen at parc.com  Tue Jun 28 18:52:19 2011
From: janssen at parc.com (Bill Janssen)
Date: Tue, 28 Jun 2011 09:52:19 PDT
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <4E09DF0C.90200@egenix.com>
References: <1309268585.21006.22.camel@marge> <4E09DF0C.90200@egenix.com>
Message-ID: <44221.1309279939@parc.com>

M.-A. Lemburg <mal at egenix.com> wrote:

> How about a more radical change: have open() in Py3 default to
> opening the file in binary mode, if no encoding is given (even
> if the mode doesn't include 'b') ?

+1.

> That'll make it compatible to the Py2 world again and avoid
> all the encoding guessing.

Yep.

Bill

From janssen at parc.com  Tue Jun 28 18:53:26 2011
From: janssen at parc.com (Bill Janssen)
Date: Tue, 28 Jun 2011 09:53:26 PDT
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <iucot6$pma$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge> <4E09DF0C.90200@egenix.com>
	<iucot6$pma$1@dough.gmane.org>
Message-ID: <44243.1309280006@parc.com>

Terry Reedy <tjreedy at udel.edu> wrote:

> > Making such default encodings depend on the locale has already
> > failed to work when we first introduced a default encoding in
> > Py2, so I don't understand why we are repeating the same
> > mistake again in Py3 (only in a different area).
> 
> I do not remember any proposed change during the Py3 design discussions.

I certainly proposed it, more than once.

Bill

From tjreedy at udel.edu  Tue Jun 28 19:06:44 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Jun 2011 13:06:44 -0400
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
Message-ID: <iud1n4$kp8$1@dough.gmane.org>

On 6/28/2011 10:46 AM, Paul Moore wrote:

> I use Windows, and come from the UK, so 99% of my text files are
> ASCII. So the majority of my code will be unaffected. But in the
> occasional situation where I use a ? sign, I'll get encoding errors,

I do not understand this. With utf-8 you would never get a string 
encoding error.

> where currently things will "just work".

As long as you only use the machine-dependent restricted character set.

 > And the failures will be data dependent, and hence intermittent
 > (the worst type of problem).

That is the situation now, with platform/machine dependencies added in.
Some people share code with other machines, even locally.

> So, in effect, you propose making the default favour writing
> multiplatform portable code at the expense of quick and dirty scripts?

Let us frame it another way. Should Python installations be compatible 
with other Python installations, or with the other apps on the same 
machine? Part of the purpose of Python is to cover up platform 
differences, to the extent possible (and perhaps sensible -- there is 
the argument). This was part of the purpose of writing our own io module 
instead of using the compiler stdlib. The evolution of floating point 
math has gone in the same direction. For instance, float now expects 
uniform platform-independent Python-dependent names for infinity and nan 
instead of compiler-dependent names.

As for practicality. Notepad++ on Windows offers ANSI, utf-8 (w,w/o 
BOM), utf-16 (big/little endian). I believe that ODF documents are utf-8 
encoded xml (compressed or not). My original claim for this proposal 
was/is that even Windows apps are moving to uft-8 and that someday 
making that the default for Python everywhere will be the obvious and 
sensible thing.

-- 
Terry Jan Reedy



From janssen at parc.com  Tue Jun 28 19:08:14 2011
From: janssen at parc.com (Bill Janssen)
Date: Tue, 28 Jun 2011 10:08:14 PDT
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <4E09D9FA.7060308@voidspace.org.uk>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<4E09D9FA.7060308@voidspace.org.uk>
Message-ID: <44642.1309280894@parc.com>

Michael Foord <fuzzyman at voidspace.org.uk> wrote:

> The new regex library has some great improvements:
> 
>     http://bugs.python.org/issue2636
> 
> It also has users and committed maintainers, so I hope we can bring it
> into 3.3. It wasn't easy to tell from skimming the change notes that
> Unicode character classes are amongst the new features. Is that the
> case?

According to http://effbot.org/zone/unicode-objects.htm (from 2004), the
existing re module already supports Unicode character classes, so the
regex module will as well.  But the support has been updated, according
to the change notes.

Bill

From fuzzyman at voidspace.org.uk  Tue Jun 28 19:22:38 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 18:22:38 +0100
Subject: [Python-Dev] open(): set the default encoding to 'utf-8'
 in	Python 3.3?
In-Reply-To: <iud1n4$kp8$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge>	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
	<iud1n4$kp8$1@dough.gmane.org>
Message-ID: <4E0A0DDE.5070403@voidspace.org.uk>

On 28/06/2011 18:06, Terry Reedy wrote:
> On 6/28/2011 10:46 AM, Paul Moore wrote:
>
>> I use Windows, and come from the UK, so 99% of my text files are
>> ASCII. So the majority of my code will be unaffected. But in the
>> occasional situation where I use a ? sign, I'll get encoding errors,
>
> I do not understand this. With utf-8 you would never get a string 
> encoding error.
>

I assumed he meant that files written out as utf-8 by python would then 
be read in using the platform encoding (i.e. not utf-8 on Windows) by 
the other applications he is inter-operating with. The error would not 
be in Python but in those applications.

>> where currently things will "just work".
>
> As long as you only use the machine-dependent restricted character set.
>

Which is the situation he is describing. You do go into those details 
below, and which choice is "correct" depends on which trade-off you want 
to make.

For the sake of backwards compatibility we are probably stuck with the 
current trade-off however - unless we deprecate using open(...) without 
an explicit encoding.

All the best,

Michael

> > And the failures will be data dependent, and hence intermittent
> > (the worst type of problem).
>
> That is the situation now, with platform/machine dependencies added in.
> Some people share code with other machines, even locally.
>
>> So, in effect, you propose making the default favour writing
>> multiplatform portable code at the expense of quick and dirty scripts?
>
> Let us frame it another way. Should Python installations be compatible 
> with other Python installations, or with the other apps on the same 
> machine? Part of the purpose of Python is to cover up platform 
> differences, to the extent possible (and perhaps sensible -- there is 
> the argument). This was part of the purpose of writing our own io 
> module instead of using the compiler stdlib. The evolution of floating 
> point math has gone in the same direction. For instance, float now 
> expects uniform platform-independent Python-dependent names for 
> infinity and nan instead of compiler-dependent names.
>
> As for practicality. Notepad++ on Windows offers ANSI, utf-8 (w,w/o 
> BOM), utf-16 (big/little endian). I believe that ODF documents are 
> utf-8 encoded xml (compressed or not). My original claim for this 
> proposal was/is that even Windows apps are moving to uft-8 and that 
> someday making that the default for Python everywhere will be the 
> obvious and sensible thing.
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From fuzzyman at voidspace.org.uk  Tue Jun 28 19:23:42 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 28 Jun 2011 18:23:42 +0100
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <44642.1309280894@parc.com>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<4E09D9FA.7060308@voidspace.org.uk> <44642.1309280894@parc.com>
Message-ID: <4E0A0E1E.2040207@voidspace.org.uk>

On 28/06/2011 18:08, Bill Janssen wrote:
> Michael Foord<fuzzyman at voidspace.org.uk>  wrote:
>
>> The new regex library has some great improvements:
>>
>>      http://bugs.python.org/issue2636
>>
>> It also has users and committed maintainers, so I hope we can bring it
>> into 3.3. It wasn't easy to tell from skimming the change notes that
>> Unicode character classes are amongst the new features. Is that the
>> case?
> According to http://effbot.org/zone/unicode-objects.htm (from 2004), the
> existing re module already supports Unicode character classes, so the
> regex module will as well.  But the support has been updated, according
> to the change notes.
Thanks. Support for Unicode character classes was one of the 
improvements needed in the re module reported from the language summit - 
so I wonder if the changes in regex are sufficient.

Michael

> Bill


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From janssen at parc.com  Tue Jun 28 19:55:45 2011
From: janssen at parc.com (Bill Janssen)
Date: Tue, 28 Jun 2011 10:55:45 PDT
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <4E0A0E1E.2040207@voidspace.org.uk>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<4E09D9FA.7060308@voidspace.org.uk> <44642.1309280894@parc.com>
	<4E0A0E1E.2040207@voidspace.org.uk>
Message-ID: <46579.1309283745@parc.com>

Michael Foord <fuzzyman at voidspace.org.uk> wrote:

> Thanks. Support for Unicode character classes was one of the
> improvements needed in the re module reported from the language summit
> - 
> so I wonder if the changes in regex are sufficient.

I guess it depends on what you're asking -- what does "support for
Unicode character classes" mean?  Apparently something different from
what it meant in 2004 :-).

Bill

From solipsis at pitrou.net  Tue Jun 28 19:56:33 2011
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 28 Jun 2011 19:56:33 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
	<iud1n4$kp8$1@dough.gmane.org>
Message-ID: <20110628195633.3b474c45@pitrou.net>

On Tue, 28 Jun 2011 13:06:44 -0400
Terry Reedy <tjreedy at udel.edu> wrote:
> 
> As for practicality. Notepad++ on Windows offers ANSI, utf-8 (w,w/o 
> BOM), utf-16 (big/little endian).

Well, that's *one* application. We would need much more data than that.

> I believe that ODF documents are utf-8 
> encoded xml (compressed or not).

XML doesn't matter for this discussion, since it explicitly declares the
encoding. What we are talking about is "raw" text files that don't have
an encoding declaration and for which the data format doesn't specify
any default encoding (which also rules out Python source code, by the
way).

> My original claim for this proposal 
> was/is that even Windows apps are moving to uft-8
> and that someday 
> making that the default for Python everywhere will be the obvious and 
> sensible thing.

True, but that may be 5 or 10 years from now.

Regards

Antoine.



From tjreedy at udel.edu  Tue Jun 28 19:58:54 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Jun 2011 13:58:54 -0400
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <4E09F4CA.3060002@voidspace.org.uk>
References: <20110626185242.GB2710@mathmagic>	<iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>	<4E08917E.3020509@btinternet.com>
	<87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
	<20110628115136.248822505A3@webabinitio.net>
	<iucrll$cfi$1@dough.gmane.org> <4E09F4CA.3060002@voidspace.org.uk>
Message-ID: <4E0A165E.5080808@udel.edu>

On 6/28/2011 11:35 AM, Michael Foord wrote:
> On 28/06/2011 16:23, Terry Reedy wrote:

>> So-called 'staticmethods' are not really methods either, but are class
>> function attributes that are just functions and not treated as
>> methods. The decorator that negates normal method treatment
>> could/should have been called 'non_method'.
>>
>> Using 'function' is its generic 'callable' sense ...
>>
>> Method: a class function attribute that in its intended and normal use
>> automagically turns the object it is called on into its first arg.
>> 'Method' is a useful and needed subcategory of class attribute
>> precisely because of this behavior.
>>
>> Instance method: a class function attribute that is (normally) called
>> on instances of the class or subclasses.

Before 2.2, these were the only methods.

> So what is the difference between "Instance method" and "Method" above?
> Is it just that "Method" is broader and includes class methods

Since 2.2, yes. The current glossary entry starts

"method A function which is defined inside a class body."

This includes 'staticmethods', but as I have said, I think that is a 
mistake. Static methods are functions without special method treatment. 
A class staticmethod function act the same as any other function.
Also, 'defined inside' is not necessary.

I would change the above to "A function that get bound to a instance or 
class when called as an attribute of the instance or class. Methods are 
usually defined inside a class body."

> and bound methods?

The result of accessing an instance or class method via an instance or 
class. Accessing a static method does not create a bound method. Bound 
methods are usually anonymous and ephemeral, being used for one call and 
then deleted.

> If anyone said "instance method" to me I would assume they meant bound
> method. (A normal method fetched from an instance.)

Instance methods are the 'permanent' class function attributes, not the 
ephemeral object that implements a.b(c). Bound methods would be an 
implementation detail, except that a.b has to evaluate to something and 
saving bound methods is quite handy when calling a method or methods 
repeatedly on the same instance.

Functools.partial is a generalization of bound methods, which were 
Python's first (special-case) implementation of the partial function 
idea. Leaving out implementation details, if b is a function attribute 
of type(a), 'a.b' is an abbreviated way of writing 
'functools.partial(type(a).b,a)' (and yes, I have tested an example of 
this).

>> Class method: a class function attribute that is (normally) called on
>> the class or subclasses.
>>
>> Bound method: a method that has already has the first-arg object
>> bundled with it, so that it can be used as a normal (partial or
>> curried) function.

---
Terry Jan Reedy

From g.brandl at gmx.net  Tue Jun 28 20:13:42 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 28 Jun 2011 20:13:42 +0200
Subject: [Python-Dev] EuroPython Language Summit report
In-Reply-To: <1309270018.21006.26.camel@marge>
References: <BANLkTinxt2QQvhmr5Zi465_YeBgvehv_Xg@mail.gmail.com>
	<20110624145516.84DA32506EA@webabinitio.net>
	<4E09D9FA.7060308@voidspace.org.uk>
	<1309270018.21006.26.camel@marge>
Message-ID: <iud5ji$c9d$1@dough.gmane.org>

On 28.06.2011 16:06, Victor Stinner wrote:
> Le mardi 28 juin 2011 ? 14:41 +0100, Michael Foord a ?crit :
>> On 24/06/2011 15:55, R. David Murray wrote:
>> > On Fri, 24 Jun 2011 10:52:40 +0200, Mark Dickinson<dickinsm at gmail.com>  wrote:
>> >> EuroPython 2011 Language Summit
>> >> ===============================
>> > [...]
>> >>      Unicode character classes is a particular need).  [Subtopic:  what needs
>> >>      to be done to get the new regex module into Python?  Should it replace
>> >>      the existing module?  What about backwards compatibility issues?]
>> > I'm pretty sure regex has backward compatibility as a goal for just this
>> > reason (so it can replace the current module).
>> >
>> The new regex library has some great improvements:
>> 
>>      http://bugs.python.org/issue2636
> 
> This issue is open since April 2008, has also the longest list of
> attached files, and has a very long history. What is the status of the
> issue? I see that there is now a third party project on:
> 
>    http://code.google.com/p/mrab-regex-hg/

This should be the same module as in the issue (and thankfully, because
code management of such a big project does not belong exclusively in a
tracker issue).  "mrab" stands for Matthew Barnett, who is the author of
regex.

> --
> 
> There is also the re2 library from Google and especially this project:
> 
>    http://pypi.python.org/pypi/re2/
> 
> "pyre2 is a Python extension that wraps Google's RE2 regular expression
> library.
> 
> This version of pyre2 is similar to the one you'd find at facebook's
> github repository except that the stated goal of *this version is to be
> a drop-in replacement for the re module*.)"

Well, while it can be called drop-in, it is hardly a good replacement:

"""
That being said, there are features of the re module that this module may never
have. For example, RE2 does not handle lookahead assertions ((?=...)).
"""

It falls back to old re in these cases.

Georg


From g.brandl at gmx.net  Tue Jun 28 20:18:00 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 28 Jun 2011 20:18:00 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <iud1n4$kp8$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
	<iud1n4$kp8$1@dough.gmane.org>
Message-ID: <iud5rk$c9d$2@dough.gmane.org>

On 28.06.2011 19:06, Terry Reedy wrote:
> On 6/28/2011 10:46 AM, Paul Moore wrote:
> 
>> I use Windows, and come from the UK, so 99% of my text files are
>> ASCII. So the majority of my code will be unaffected. But in the
>> occasional situation where I use a ? sign, I'll get encoding errors,
> 
> I do not understand this. With utf-8 you would never get a string 
> encoding error.

Yes, but you'll get plenty of *decoding* errors.

Georg


From ethan at stoneleaf.us  Tue Jun 28 20:55:02 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 28 Jun 2011 11:55:02 -0700
Subject: [Python-Dev] open(): set the default encoding to 'utf-8'
 in	Python 3.3?
In-Reply-To: <4E0A0655.40207@voidspace.org.uk>
References: <1309268585.21006.22.camel@marge>	<4E09DF0C.90200@egenix.com>	<iucot6$pma$1@dough.gmane.org>	<4E09E9A9.5020702@voidspace.org.uk>	<iucvrg$885$1@dough.gmane.org>
	<4E0A0655.40207@voidspace.org.uk>
Message-ID: <4E0A2386.1090502@stoneleaf.us>

Michael Foord wrote:
> On 28/06/2011 17:34, Terry Reedy wrote:
>> On 6/28/2011 10:48 AM, Michael Foord wrote:
>>> On 28/06/2011 15:36, Terry Reedy wrote:
>>
>>>> S = open('myfile.txt').read()
>>>> now return a text string in both Py2 and Py3 and a subsequent
>>>> 'abc' in S
>>>> works in both.
>>>
>>> Nope, it returns a bytestring in Python 2.
>>
>> Which, in Py2 is a str() object. 
> 
> Yes, but not a "text string". The equivalent of the Python 2 str in 
> Python 3 is bytes. Irrelevant discussion anyway.

Irrelevant to the OP, yes, but a Python 2 string *is not* the same as 
Python 3 bytes.  If you don't believe me fire up your Python 3 shell and 
try b'xyz'[1] == 'y'.

~Ethan~

From ethan at stoneleaf.us  Tue Jun 28 21:06:01 2011
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 28 Jun 2011 12:06:01 -0700
Subject: [Python-Dev] open(): set the default encoding to 'utf-8'
 in	Python 3.3?
In-Reply-To: <4E0A2386.1090502@stoneleaf.us>
References: <1309268585.21006.22.camel@marge>	<4E09DF0C.90200@egenix.com>	<iucot6$pma$1@dough.gmane.org>	<4E09E9A9.5020702@voidspace.org.uk>	<iucvrg$885$1@dough.gmane.org>	<4E0A0655.40207@voidspace.org.uk>
	<4E0A2386.1090502@stoneleaf.us>
Message-ID: <4E0A2619.8020605@stoneleaf.us>

Ethan Furman wrote:
> Michael Foord wrote:
>> On 28/06/2011 17:34, Terry Reedy wrote:
>>> On 6/28/2011 10:48 AM, Michael Foord wrote:
>>>> On 28/06/2011 15:36, Terry Reedy wrote:
>>>
>>>>> S = open('myfile.txt').read()
>>>>> now return a text string in both Py2 and Py3 and a subsequent
>>>>> 'abc' in S
>>>>> works in both.
>>>>
>>>> Nope, it returns a bytestring in Python 2.
>>>
>>> Which, in Py2 is a str() object. 
>>
>> Yes, but not a "text string". The equivalent of the Python 2 str in 
>> Python 3 is bytes. Irrelevant discussion anyway.
> 
> Irrelevant to the OP, yes, but a Python 2 string *is not* the same as 
> Python 3 bytes.  If you don't believe me fire up your Python 3 shell and 
> try b'xyz'[1] == 'y'.

er, make that b'xyz'[1] == b'y'  :(

From brett at python.org  Tue Jun 28 21:05:39 2011
From: brett at python.org (Brett Cannon)
Date: Tue, 28 Jun 2011 12:05:39 -0700
Subject: [Python-Dev] Issue10403 - using 'attributes' instead of members
 in documentation
In-Reply-To: <BANLkTikOKo-nAQ-gtUJ75pUxkaPTVCQo1A@mail.gmail.com>
References: <20110626185242.GB2710@mathmagic> <iu89n5$eg3$1@dough.gmane.org>
	<BANLkTi=0TzsuO+wd_6v5+z3PO5_SXPOq6Q@mail.gmail.com>
	<20110627102428.6fc1955b@msiwind>
	<BANLkTi=OuxgzOWT+1M0QFJA0gKWYRsO_jg@mail.gmail.com>
	<20110627140826.877032505A3@webabinitio.net>
	<4E08917E.3020509@btinternet.com> <87r56en1rt.fsf@benfinney.id.au>
	<BANLkTinPeJD=Ug9pVyO+4V=JN8pNKw1FoQ@mail.gmail.com>
	<87mxh2mtq8.fsf@benfinney.id.au>
	<BANLkTi=n2CyqYZGmDnpjxHaL1=T7HAL8jg@mail.gmail.com>
	<BANLkTi=ffjgBnbaRwOXAunRA6LHzygeVJg@mail.gmail.com>
	<4E09B2EF.9030304@voidspace.org.uk>
	<BANLkTimi_=T1AcPn-uMGYgnt90Lwag1OGA@mail.gmail.com>
	<4E09CF88.4090307@voidspace.org.uk>
	<BANLkTikOKo-nAQ-gtUJ75pUxkaPTVCQo1A@mail.gmail.com>
Message-ID: <BANLkTi=yXK5Gg1HVod5jEXnQSkQEbuNE0kkWgicCdkgoZQynVQ@mail.gmail.com>

On Tue, Jun 28, 2011 at 06:28, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Tue, Jun 28, 2011 at 10:56 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
> > I don't think "data attributes" is clear or precise. Is a property a data
> > attribute (well it depends how it is implemented and what it does), is a
> > staticmethod a data attribute (no - but then Tres' question - it isn't a
> > normal method either so if you define data attribute to mean "all non
> method
> > attributes" then its potentially an open question).
>
> "callable attributes" and "data attributes" are purely about the API
> exposed by the object in question. If we're going for object model
> neutral terminology, that's the only view that makes sense. They're
> descriptions about how attributes are intended to be *used* that are
> completely silent on the question of how they're *implemented*. So
> staticmethod would fall into the first group, while property would
> fall into the latter.
>

These two terms also have the benefit of being understandable by non Python
programmers. Since we can't rely on people read the glossary or any other
specific doc before reading the stdlib docs we need to make sure that the
terminology that we use can be understood by newbies. Thus I like data and
callable attributes since it makes sense even if you come from another
programming language since you can easily reason it out.

And now I am done adding my graffiti to the shed.

-Brett


>
> >From an implementation point of view, you carve up the world
> differently, so it makes sense to worry about class attributes,
> instance attributes, dynamic attributes, etc. (the class vs instance
> distinction can also matter to some degree from the usage point of
> view, since it affects the scope of any mutable attributes, and the
> static vs dynamic distinction can also matter, especially for
> introspection purposes).
>
> This goes back to the original point about all of this being highly
> context dependent - how you carve up the set of all attributes is
> going to change based on what you're trying to explain (e.g. the
> distinction between enforced 'data' descriptors, descriptors that
> allow shadowing in the instance dict, class attributes that aren't
> descriptors at all, instance attributes and dynamic attributes
> retrieved via __getattr__ is another way of dividing them)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110628/1a3fb3f7/attachment.html>

From p.f.moore at gmail.com  Tue Jun 28 22:11:50 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 28 Jun 2011 21:11:50 +0100
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <4E0A0DDE.5070403@voidspace.org.uk>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
	<iud1n4$kp8$1@dough.gmane.org> <4E0A0DDE.5070403@voidspace.org.uk>
Message-ID: <BANLkTimDdPVNYP9ZJXVrDa=POOH1Aj+Lfw@mail.gmail.com>

On 28 June 2011 18:22, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> On 28/06/2011 18:06, Terry Reedy wrote:
>>
>> On 6/28/2011 10:46 AM, Paul Moore wrote:
>>
>>> I use Windows, and come from the UK, so 99% of my text files are
>>> ASCII. So the majority of my code will be unaffected. But in the
>>> occasional situation where I use a ? sign, I'll get encoding errors,
>>
>> I do not understand this. With utf-8 you would never get a string encoding
>> error.
>>
>
> I assumed he meant that files written out as utf-8 by python would then be
> read in using the platform encoding (i.e. not utf-8 on Windows) by the other
> applications he is inter-operating with. The error would not be in Python
> but in those applications.

That is correct. Or files written out (as platform encoding) by other
applications, will later be read in as UTF-8 by Python, and be seen as
incorrect characters, or worse raise decoding errors. (Sorry, in my
original post I said "encoding" where I meant "decoding"...)

I'm not interested in allocating "blame" for the "error". I'm not
convinced that it *is* an error, merely 2 programs with incompatible
assumptions. What I'm saying is that compatibility between various
programs on a single machine can, in some circumstances, be more
important than compatibility between (the same, or different) programs
running on different machines or OSes. And that I, personally, am in
that situation.

>>> where currently things will "just work".
>>
>> As long as you only use the machine-dependent restricted character set.
>>
>
> Which is the situation he is describing. You do go into those details below,
> and which choice is "correct" depends on which trade-off you want to make.
>
> For the sake of backwards compatibility we are probably stuck with the
> current trade-off however - unless we deprecate using open(...) without an
> explicit encoding.

Backward compatibility is another relevant point. But other than that,
it's a design trade-off, agreed. All I'm saying is that I see the
current situation (which is in favour of quick script use and beginner
friendly at the expense of conceptual correctness and forcing the user
to think about his choices) as being preferable (and arguably more
"Pythonic", in the sense that I see it as a case of "practicality
beats purity" - although it's easy to argue that "in the face of
ambiguity..." also applies here :-))

Paul.

From victor.stinner at haypocalc.com  Tue Jun 28 22:51:38 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 28 Jun 2011 22:51:38 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <4E09DF0C.90200@egenix.com>
References: <1309268585.21006.22.camel@marge>  <4E09DF0C.90200@egenix.com>
Message-ID: <1309294298.23387.12.camel@marge>

Le mardi 28 juin 2011 ? 16:02 +0200, M.-A. Lemburg a ?crit :
> How about a more radical change: have open() in Py3 default to
> opening the file in binary mode, if no encoding is given (even
> if the mode doesn't include 'b') ?

I tried your suggested change: Python doesn't start.

sysconfig uses the implicit locale encoding to read sysconfig.cfg, the
Makefile and pyconfig.h. I think that it is correct to use the locale
encoding for Makefile and pyconfig.h, but maybe not for sysconfig.cfg.

Python require more changes just to run "make". I was able to run "make"
by using encoding='utf-8' in various functions (of distutils and
setup.py). I didn't try the test suite, I expect too many failures.

--

Then I tried my suggestion (use "utf-8" by default): Python starts
correctly, I can build it (run "make") and... the full test suite pass
without any change. (I'm testing on Linux, my locale encoding is UTF-8.)

Victor


From g.brandl at gmx.net  Tue Jun 28 23:42:11 2011
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 28 Jun 2011 21:42:11 +0000
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <iuco7n$k7r$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge> <iuco7n$k7r$1@dough.gmane.org>
Message-ID: <iudhql$ql1$1@dough.gmane.org>

On 28.06.2011 14:24, Terry Reedy wrote:

>> As discussed before on this list, I propose to set the default encoding
>> of open() to UTF-8 in Python 3.3, and add a warning in Python 3.2 if
>> open() is called without an explicit encoding and if the locale encoding
>> is not UTF-8. Using the warning, you will quickly notice the potential
>> problem (using Python 3.2.2 and -Werror) on Windows or by using a
>> different locale encoding (.e.g using LANG="C").

[...]

>> Should I convert this email into a PEP, or is it not required?
> 
> I think a PEP is needed.

Absolutely.  And I hope the hypothetical PEP would be rejected in this form.

We need to stop making incompatible changes to Python 3.  We had the chance
and took it to break all kinds of stuff, some of it gratuitous, with 3.0 and
even 3.1.  Now the users need a period of compatibility and stability (just
like the language moratorium provided for one aspect of Python).

Think about porting: Python 3 uptake is not ahead of time (I don't want to
say it's too slow, but it's certainly not too fast.)  For the sake of porters'
sanity, 3.x should not be a moving target.  New features are not so much of
a problem, but incompatibilities like this one certainly are.

At the very least, a change like this needs a transitional strategy, like
it has been used during the 2.x series:

* In 3.3, accept "locale" as the encoding parameter, meaning the locale encoding
* In 3.4, warn if encoding isn't given and the locale encoding isn't UTF-8
* In 3.5, change default encoding to UTF-8

It might be just enough to stress in the documentation that usage of the
encoding parameter is recommended for cross-platform consistency.

cheers,
Georg


From victor.stinner at haypocalc.com  Wed Jun 29 00:00:45 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Jun 2011 00:00:45 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
Message-ID: <1309298445.23387.33.camel@marge>

> > I don't think that Windows developer even know that they are writing
> > files into the ANSI code page. MSDN documentation of
> > WideCharToMultiByte() warns developer that the ANSI code page is not
> > portable, even accross Windows computers:
> 
> Probably true. But for many uses they also don't care. If you're
> writing something solely for a one-off job on your own PC, the ANSI
> code page is fine, and provides interoperability with other programs
> on your PC, which is really what you care about. (UTF-8 without BOM
> displays incorrectly in Vim, wordpad, and powershell get-content.

I tried to open a text file encoded to UTF-8 (without BOM) on Windows
Seven.

The default application displays it correctly, it's the well known
builtin notepad program.

gvim is unable to detect the encoding, it reads the file using the ANSI
code page (WTF? UTF-8 is correctly detected on Linux!?). 

Wordpad reads the file using the ANSI code page, it is unable to detect
the UTF-8 encoding.

The "type" command in a MS-Dos shell (cmd.exe) dosen't display the UTF-8
correctly, but a file encoded to ANSI code is also display incorrectly.
I suppose that the problem is that the terminal uses the OEM code page,
not the ANSI code page.

Visual C++ 2008 detects the UTF-8 encoding.

I don't have other applications to test on my Windows Seven. I agree
that UTF-8 is not well supported by "standard" Windows applications. I
would at least expect that Wordpad and gvim are able to detect the UTF-8
encoding.

> MBCS works fine in all of these. It also displays incorrectly in CMD type,
> but in a less familiar form than the incorrect display mbcs produces,
> for what that's worth...)

True, the encoding of a text file encoded to the ANSI code page is
correctly detected by all applications (except "type" in a shell, it
should be the OEM/ANSI code page conflict).

> IMHO, you missed another option - open() does not need improving, the
> current behaviour is better than any of the 3 options noted.

My original need is to detect that my program will behave differently on
Linux and Windows, because open() uses the implicit locale encoding.
Antoine suggested me to monkeypatch __builtins__.open to do that.

Victor


From tseaver at palladion.com  Wed Jun 29 00:29:16 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 28 Jun 2011 18:29:16 -0400
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <44221.1309279939@parc.com>
References: <1309268585.21006.22.camel@marge> <4E09DF0C.90200@egenix.com>
	<44221.1309279939@parc.com>
Message-ID: <iudkjs$74p$2@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/28/2011 12:52 PM, Bill Janssen wrote:
> M.-A. Lemburg <mal at egenix.com> wrote:
> 
>> How about a more radical change: have open() in Py3 default to
>> opening the file in binary mode, if no encoding is given (even
>> if the mode doesn't include 'b') ?
> 
> +1.
> 
>> That'll make it compatible to the Py2 world again and avoid
>> all the encoding guessing.
> 
> Yep.

+1 from me, as well:  "in the face of ambiguity, refuse the temptation
sto guess."


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4KVbwACgkQ+gerLs4ltQ4gPACgoWGjAhmOg9IGQgMht2KsZYn5
mKUAnjcLP6BGCVFSudm1v77ZHere3VHw
=4SPu
-----END PGP SIGNATURE-----


From victor.stinner at haypocalc.com  Wed Jun 29 00:34:54 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Jun 2011 00:34:54 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <20110628163351.GK27252@unaka.lan>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
	<20110628163351.GK27252@unaka.lan>
Message-ID: <1309300494.23387.49.camel@marge>

Le mardi 28 juin 2011 ? 09:33 -0700, Toshio Kuratomi a ?crit :
> Issuing a warning like "open used without explicit encoding may lead
> to errors" if open() is used without an explicit encoding would help
> a little (at least, people who get errors would then have an inkling
> that the culprit might be an open() call).  If I read Victor's previous
> email correctly, though, he said this was previously rejected.

Oh sorry, I used the wrong word. I listed two other possible solutions,
but there were not really rejetected. I just thaugh that changing the
default encoding to UTF-8 was the most well accepted idea.

If I mix different suggestions together: another solution is to emit a
warning if the encoding is not specified (not only if the locale
encoding is different than UTF-8). Using encoding="locale" would make it
quiet. It would be annoying if the warning would be displayed by default
("This will make things harder for simple scripts which are not
intended to be cross-platform." wrote Paul Moore). It only makes sense
if we use the same policy than unclosed files/sockets: hidden by
default, but it can be configured using command line options (-Werror,
yeah!).

> Another brainstorming solution would be to use different default encodings on
> different platforms.  For instance, for writing files, utf-8 on *nix systems
> (including macosX) and utf-16 on windows.

I don't think that UTF-16 is a better choice than UTF-8 on Windows :-(

> For reading files, check for a utf-16 BOM, if not present, operate as utf-8.

Oh oh. I already suggested to read the BOM. See
http://bugs.python.org/issue7651 and read the email thread "Improve
open() to support reading file starting with an unicode BOM"
http://mail.python.org/pipermail/python-dev/2010-January/097102.html

Reading the BOM is a can of worm, everybody expects something different.
I forgot the idea of changing that.

Victor


From tjreedy at udel.edu  Wed Jun 29 02:22:59 2011
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 28 Jun 2011 20:22:59 -0400
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <iudhql$ql1$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge> <iuco7n$k7r$1@dough.gmane.org>
	<iudhql$ql1$1@dough.gmane.org>
Message-ID: <iudr92$bdk$1@dough.gmane.org>

On 6/28/2011 5:42 PM, Georg Brandl wrote:

> At the very least, a change like this needs a transitional strategy, like
> it has been used during the 2.x series:
>
> * In 3.3, accept "locale" as the encoding parameter, meaning the locale encoding
> * In 3.4, warn if encoding isn't given and the locale encoding isn't UTF-8
> * In 3.5, change default encoding to UTF-8

3.5 should be 4-5 years off. I actually would not propose anything 
faster than that.

-- 
Terry Jan Reedy


From rndblnch at gmail.com  Wed Jun 29 02:34:59 2011
From: rndblnch at gmail.com (renaud)
Date: Wed, 29 Jun 2011 00:34:59 +0000 (UTC)
Subject: [Python-Dev] PEP 380 acceptance (was Re: EuroPython Language
	Summit report)
References: <BANLkTin1Voeg-7hBUOSwrMQNwu9BqE-nQA@mail.gmail.com>
	<BANLkTikzx4w3GS9FfHM+q4GQ1Ur-bU5L_A@mail.gmail.com>
	<loom.20110627T170547-215@post.gmane.org>
	<BANLkTimMMg+DHbkuXFdVtqiRcCFVL06dew@mail.gmail.com>
Message-ID: <loom.20110629T022622-661@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> 
> On Tue, Jun 28, 2011 at 1:09 AM, renaud <rndblnch <at> gmail.com> wrote:
> > Nick Coghlan <ncoghlan <at> gmail.com> writes:
> >
> >> I hit a snag with this. The real tests of the PEP 380 functionality
> >> aren't currently part of the patch - they're a big set of "golden
> >> output" tests in the zipfile hosted on Greg's site. Those need to be
> >> refactored into proper unittest or doctest based additions to the test
> >> suite and incorporated into the patch before I could commit this with
> >> a clear conscience.
> >
> > let me know if i can help.
> 
> It would be good if you could take a look at Greg's original test
> suite, consider ways of bringing it into the main regression tests and
> then update the patch queue on bitbucket accordingly.
> 
> My preference is for something unittest based, essentially taking the
> "golden output" comparisons and turning them into appropriate
> self.assert* invocations.
> 
> Given the number of tests Greg has, it will probably make more sense
> to do it as a new test subdirectory rather than as a single test file
> (although that depends on how many tests are in each file - if there
> are only a few, or if they overlap a lot, then having them as separate
> test cases within a single file may be a better choice).

ok, i've generated a single test_pep380.py using greg tests wrapped to be
run by unittest.
it's ugly, but it does the job.
some things that may not be desirable:
- line numbers in the tracebacks do rely on the position of the tests in the 
file, so any editing before the last test case will probably screw up 
everything;
- the test touches sys.stdout & sys.stderr

the whole thing is available here:
<https://bitbucket.org/rndblnch/cpython-pep380/src/tip/pep380-tests>

renaud

ps. i had to edit the test24 (Test parser module) expected output to make it
match the actual output, i guess that the parser module has changed since greg
wrote the tests.



From vinay_sajip at yahoo.co.uk  Wed Jun 29 03:05:18 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 29 Jun 2011 01:05:18 +0000 (UTC)
Subject: [Python-Dev] svn.python.org confusion
References: <nad-47439B.09372828062011@news.gmane.org>
Message-ID: <loom.20110629T030205-927@post.gmane.org>

Ned Deily <nad <at> acm.org> writes:

> Could some text be added to the svn browser pages to at least indicate 
> that the repo is no longer being updated with a link to the 
> hg.python.org browser?   I'm not sure what to do about the repos 
> themselves if people attempt to do an svn co.  Perhaps that should just 
> be disabled totally for python?

At least some of the stuff in the SVN repo is still needed, AFAICT. I recently
did a build of the Python Windows binary installer, and I understand (from the
buildbot instructions) that the best source of some of the external dependencies
(bz2, Tcl/Tk, openssl etc.) is still the SVN repo.

Regards,

Vinay Sajip


From stefan_ml at behnel.de  Wed Jun 29 08:28:41 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 29 Jun 2011 08:28:41 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <1309268585.21006.22.camel@marge>
References: <1309268585.21006.22.camel@marge>
Message-ID: <iuegmp$98r$1@dough.gmane.org>

Victor Stinner, 28.06.2011 15:43:
> In Python 2, open() opens the file in binary mode (e.g. file.readline()
> returns a byte string). codecs.open() opens the file in binary mode by
> default, you have to specify an encoding name to open it in text mode.
>
> In Python 3, open() opens the file in text mode by default. (It only
> opens the binary mode if the file mode contains "b".) The problem is
> that open() uses the locale encoding if the encoding is not specified,
> which is the case *by default*. The locale encoding can be:
>
>   - UTF-8 on Mac OS X, most Linux distributions
>   - ISO-8859-1 os some FreeBSD systems
>   - ANSI code page on Windows, e.g. cp1252 (close to ISO-8859-1) in
> Western Europe, cp952 in Japan, ...
>   - ASCII if the locale is manually set to an empty string or to "C", or
> if the environment is empty, or by default on some systems
>   - something different depending on the system and user configuration...
>
> If you develop under Mac OS X or Linux, you may have surprises when you
> run your program on Windows on the first non-ASCII character. You may
> not detect the problem if you only write text in english... until
> someone writes the first letter with a diacritic.

I agree that this is a *very* common source of problems. People write code 
that doesn't care about encodings all over the place, and are then 
surprised when it stops working at some point, either by switching 
environments or by changing the data. I've seen this in virtually all 
projects I've ever come to work in[1]. So, eventually, all of that code was 
either thrown away or got debugged and fixed to use an explicit (and 
usually configurable) encoding.

Consequently, I don't think it's a bad idea to break out of this ever 
recurring development cycle by either requiring an explicit encoding right 
from the start, or by making the default encoding platform independent. The 
opportunity to fix this was very unfortunately missed in Python 3.0.

Personally, I don't buy the argument that it's harder to write quick 
scripts if an explicit encoding is required. Most code that gets written is 
not just quick scripts, and even those tend to live longer than initially 
intended.

Stefan


[1] Admittedly, most of those projects were in Java, where the situation is 
substantially worse than in Python. Java entirely lacks a way to define a 
per-module source encoding, and it even lacks a straight forward way to 
encode/decode a file with an explicit encoding. So, by default, *both* 
input encodings are platform dependent, whereas in Python it's only the 
default file encoding, and properly decoding a file is straight forward there.


From ncoghlan at gmail.com  Wed Jun 29 08:52:37 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 29 Jun 2011 16:52:37 +1000
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <iudhql$ql1$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge> <iuco7n$k7r$1@dough.gmane.org>
	<iudhql$ql1$1@dough.gmane.org>
Message-ID: <BANLkTin-j66=zah+f+hMX3P5mPpOoeWsXQ@mail.gmail.com>

On Wed, Jun 29, 2011 at 7:42 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> On 28.06.2011 14:24, Terry Reedy wrote:
>> I think a PEP is needed.
>
> Absolutely. ?And I hope the hypothetical PEP would be rejected in this form.
>
> We need to stop making incompatible changes to Python 3. ?We had the chance
> and took it to break all kinds of stuff, some of it gratuitous, with 3.0 and
> even 3.1. ?Now the users need a period of compatibility and stability (just
> like the language moratorium provided for one aspect of Python).

+1 to everything Georg said.

- nothing can change in 3.2
- perhaps provide a way for an application to switch the default
behaviour between 'locale' and 'utf-8' in 3.3
- if this is done, also provide a way to explicitly request the
'locale' behaviour (likely via a locale dependent codec alias)
- maybe start thinking about an actual transition to 'utf-8' as
default in the 3.4/5 time frame

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From barry at python.org  Wed Jun 29 08:53:39 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 29 Jun 2011 07:53:39 +0100
Subject: [Python-Dev] svn.python.org confusion
In-Reply-To: <loom.20110629T030205-927@post.gmane.org>
References: <nad-47439B.09372828062011@news.gmane.org>
	<loom.20110629T030205-927@post.gmane.org>
Message-ID: <20110629075339.4bd05b1e@snowdog>

On Jun 29, 2011, at 01:05 AM, Vinay Sajip wrote:

>Ned Deily <nad <at> acm.org> writes:
>
>> Could some text be added to the svn browser pages to at least indicate 
>> that the repo is no longer being updated with a link to the 
>> hg.python.org browser?   I'm not sure what to do about the repos 
>> themselves if people attempt to do an svn co.  Perhaps that should just 
>> be disabled totally for python?
>
>At least some of the stuff in the SVN repo is still needed, AFAICT. I recently
>did a build of the Python Windows binary installer, and I understand (from the
>buildbot instructions) that the best source of some of the external dependencies
>(bz2, Tcl/Tk, openssl etc.) is still the SVN repo.

In fact, it's also not entirely true that svn isn't being updated.  We're
still cutting any 2.5 and 2.6 releases from svn, though we're also porting the
changes into hg.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110629/0798d5dd/attachment.pgp>

From sandro.tosi at gmail.com  Wed Jun 29 08:59:01 2011
From: sandro.tosi at gmail.com (Sandro Tosi)
Date: Wed, 29 Jun 2011 08:59:01 +0200
Subject: [Python-Dev] svn.python.org confusion
In-Reply-To: <loom.20110629T030205-927@post.gmane.org>
References: <nad-47439B.09372828062011@news.gmane.org>
	<loom.20110629T030205-927@post.gmane.org>
Message-ID: <BANLkTik4FKv-=704ua0SmdFj4ab4X5UVdQ@mail.gmail.com>

On Wed, Jun 29, 2011 at 03:05, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Ned Deily <nad <at> acm.org> writes:
>
>> Could some text be added to the svn browser pages to at least indicate
>> that the repo is no longer being updated with a link to the
>> hg.python.org browser? ? I'm not sure what to do about the repos
>> themselves if people attempt to do an svn co. ?Perhaps that should just
>> be disabled totally for python?
>
> At least some of the stuff in the SVN repo is still needed, AFAICT. I recently
> did a build of the Python Windows binary installer, and I understand (from the
> buildbot instructions) that the best source of some of the external dependencies
> (bz2, Tcl/Tk, openssl etc.) is still the SVN repo.

SVN is also used to fetch third-party tools to build documentation,
such as: sphinx, docutils, jinja, and pygments. These locations are
also advertised on
http://docs.python.org/py3k/documenting/building.html (that contains
outdated info, but that's another story).

Cheers,
-- 
Sandro Tosi (aka morph, morpheus, matrixhasu)
My website: http://matrixhasu.altervista.org/
Me at Debian: http://wiki.debian.org/SandroTosi

From devel at baptiste-carvello.net  Wed Jun 29 09:21:19 2011
From: devel at baptiste-carvello.net (Baptiste Carvello)
Date: Wed, 29 Jun 2011 09:21:19 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
	Python 3.3?
In-Reply-To: <BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
Message-ID: <iuejsq$pl7$1@dough.gmane.org>

Le 28/06/2011 16:46, Paul Moore a ?crit :
>
> -1. This will make things harder for simple scripts which are not
> intended to be cross-platform.
>
+1 to all you said.

I frequently use the python command prompt or "python -c" for various quick 
tasks (mostly on linux). I would hate to replace my ugly, but working

 >>> open('example.txt').read()

with the unnecessarily verbose

 >>> open('example.txt',encoding='utf-8').read()

When using python that way as a "swiss army knife", typing does matter.


My preferred solution would be:

>> - emit a warning if the encoding argument is not set

By the way, I just thought that for real programming, I would love to have a 
-Wcrossplatform command switch, which would warn for all unportable constructs 
in one go. That way, I don't have to remember which parts of 'os' wrap 
posix-only functionnality.

Baptiste


From barry at python.org  Wed Jun 29 09:58:51 2011
From: barry at python.org (Barry Warsaw)
Date: Wed, 29 Jun 2011 08:58:51 +0100
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <iudhql$ql1$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge> <iuco7n$k7r$1@dough.gmane.org>
	<iudhql$ql1$1@dough.gmane.org>
Message-ID: <20110629085851.3df6f39b@snowdog>

On Jun 28, 2011, at 09:42 PM, Georg Brandl wrote:

>We need to stop making incompatible changes to Python 3.  We had the chance
>and took it to break all kinds of stuff, some of it gratuitous, with 3.0 and
>even 3.1.  Now the users need a period of compatibility and stability (just
>like the language moratorium provided for one aspect of Python).

+1.  I think this is the #1 complaint I hear about Python in talking to
users.  I think in general we do a pretty good job of maintaining backward
compatibility between releases, but not a perfect job, and the places where we
miss can be painful for folks.  It may be difficult to achieve in all cases,
but compatibility should be carefully and thoroughly considered for all
changes, especially in the stdlib, and clearly documented where deliberate
decisions to break that are adopted.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110629/a57eee01/attachment.pgp>

From mal at egenix.com  Wed Jun 29 10:18:59 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 29 Jun 2011 10:18:59 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8'
 in	Python 3.3?
In-Reply-To: <1309294298.23387.12.camel@marge>
References: <1309268585.21006.22.camel@marge> <4E09DF0C.90200@egenix.com>
	<1309294298.23387.12.camel@marge>
Message-ID: <4E0ADFF3.50100@egenix.com>

Victor Stinner wrote:
> Le mardi 28 juin 2011 ? 16:02 +0200, M.-A. Lemburg a ?crit :
>> How about a more radical change: have open() in Py3 default to
>> opening the file in binary mode, if no encoding is given (even
>> if the mode doesn't include 'b') ?
> 
> I tried your suggested change: Python doesn't start.

No surprise there: it's an incompatible change, but one that undoes
a wart introduced in the Py3 transition. Guessing encodings should
be avoided whenever possible.

> sysconfig uses the implicit locale encoding to read sysconfig.cfg, the
> Makefile and pyconfig.h. I think that it is correct to use the locale
> encoding for Makefile and pyconfig.h, but maybe not for sysconfig.cfg.
> 
> Python require more changes just to run "make". I was able to run "make"
> by using encoding='utf-8' in various functions (of distutils and
> setup.py). I didn't try the test suite, I expect too many failures.

This demonstrates that Python's stdlib is still not being explicit
about the encoding issues. I suppose that things just happen to work
because we mostly use ASCII files for configuration and setup.

> --
> 
> Then I tried my suggestion (use "utf-8" by default): Python starts
> correctly, I can build it (run "make") and... the full test suite pass
> without any change. (I'm testing on Linux, my locale encoding is UTF-8.)

I bet it would also with "ascii" in most cases. Which then just
means that the Python build process and test suite is not a good
test case for choosing a default encoding.

Linux is also a poor test candidate for this, since most user setups
will use UTF-8 as locale encoding. Windows, OTOH, uses all sorts of
code page encodings (usually not UTF-8), so you are likely to hit
the real problem cases a lot easier.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 29 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From nad at acm.org  Wed Jun 29 10:25:42 2011
From: nad at acm.org (Ned Deily)
Date: Wed, 29 Jun 2011 01:25:42 -0700
Subject: [Python-Dev] svn.python.org confusion
References: <nad-47439B.09372828062011@news.gmane.org>
	<loom.20110629T030205-927@post.gmane.org>
	<BANLkTik4FKv-=704ua0SmdFj4ab4X5UVdQ@mail.gmail.com>
Message-ID: <nad-7156D6.01254229062011@news.gmane.org>

In article <BANLkTik4FKv-=704ua0SmdFj4ab4X5UVdQ at mail.gmail.com>,
 Sandro Tosi <sandro.tosi at gmail.com> wrote:
> On Wed, Jun 29, 2011 at 03:05, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> > Ned Deily <nad <at> acm.org> writes:
> >> Could some text be added to the svn browser pages to at least indicate
> >> that the repo is no longer being updated with a link to the
> >> hg.python.org browser? ? I'm not sure what to do about the repos
> >> themselves if people attempt to do an svn co. ?Perhaps that should just
> >> be disabled totally for python?
> > At least some of the stuff in the SVN repo is still needed, AFAICT. I 
> > recently
> > did a build of the Python Windows binary installer, and I understand (from 
> > the
> > buildbot instructions) that the best source of some of the external 
> > dependencies
> > (bz2, Tcl/Tk, openssl etc.) is still the SVN repo.
> 
> SVN is also used to fetch third-party tools to build documentation,
> such as: sphinx, docutils, jinja, and pygments. These locations are
> also advertised on
> http://docs.python.org/py3k/documenting/building.html (that contains
> outdated info, but that's another story).

Yes, by "for python", I meant the active python development "branches" 
(py3k->default, 3.2, 2.7).

-- 
 Ned Deily,
 nad at acm.org


From victor.stinner at haypocalc.com  Wed Jun 29 11:50:57 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Jun 2011 11:50:57 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <4E0ADFF3.50100@egenix.com>
References: <1309268585.21006.22.camel@marge>  <4E09DF0C.90200@egenix.com>
	<1309294298.23387.12.camel@marge>  <4E0ADFF3.50100@egenix.com>
Message-ID: <1309341057.8003.9.camel@marge>

Le mercredi 29 juin 2011 ? 10:18 +0200, M.-A. Lemburg a ?crit :
> Victor Stinner wrote:
> > Le mardi 28 juin 2011 ? 16:02 +0200, M.-A. Lemburg a ?crit :
> >> How about a more radical change: have open() in Py3 default to
> >> opening the file in binary mode, if no encoding is given (even
> >> if the mode doesn't include 'b') ?
> > 
> > I tried your suggested change: Python doesn't start.
> 
> No surprise there: it's an incompatible change, but one that undoes
> a wart introduced in the Py3 transition. Guessing encodings should
> be avoided whenever possible.

It means that all programs written for Python 3.0, 3.1, 3.2 will stop
working with the new 3.x version (let say 3.3). Users will have to
migrate from Python 2 to Python 3.2, and then migration from Python 3.2
to Python 3.3 :-(

I would prefer a ResourceWarning (emited if the encoding is not
specified), hidden by default: it doesn't break compatibility, and
-Werror gives exactly the same behaviour that you expect.

> This demonstrates that Python's stdlib is still not being explicit
> about the encoding issues. I suppose that things just happen to work
> because we mostly use ASCII files for configuration and setup.

I did more tests. I found some mistakes and sometimes the binary mode
can be used, but most function really expect the locale encoding (it is
the correct encoding to read-write files). I agree that it would be to
have an explicit encoding="locale", but make it mandatory is a little
bit rude.

> > Then I tried my suggestion (use "utf-8" by default): Python starts
> > correctly, I can build it (run "make") and... the full test suite pass
> > without any change. (I'm testing on Linux, my locale encoding is UTF-8.)
> 
> I bet it would also with "ascii" in most cases. Which then just
> means that the Python build process and test suite is not a good
> test case for choosing a default encoding.
> 
> Linux is also a poor test candidate for this, since most user setups
> will use UTF-8 as locale encoding. Windows, OTOH, uses all sorts of
> code page encodings (usually not UTF-8), so you are likely to hit
> the real problem cases a lot easier.

I also ran the test suite on my patched Python (open uses UTF-8 by
default) with ASCII locale encoding (LANG=C), the test suite does also
pass. Many tests uses non-ASCII characters, some of them are skipped if
the locale encoding is unable to encode the tested text.

Victor


From victor.stinner at haypocalc.com  Wed Jun 29 12:01:28 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Jun 2011 12:01:28 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8' in
 Python 3.3?
In-Reply-To: <iuejsq$pl7$1@dough.gmane.org>
References: <1309268585.21006.22.camel@marge>
	<BANLkTin1dRp1Dr57tfFaVwzjRKOjqDHfHA@mail.gmail.com>
	<iuejsq$pl7$1@dough.gmane.org>
Message-ID: <1309341688.8003.19.camel@marge>

Le mercredi 29 juin 2011 ? 09:21 +0200, Baptiste Carvello a ?crit :
> By the way, I just thought that for real programming, I would love to have a 
> -Wcrossplatform command switch, which would warn for all unportable constructs 
> in one go. That way, I don't have to remember which parts of 'os' wrap 
> posix-only functionnality.

When I developed using PHP, error_reporting(E_ALL) was really useful. I
would like a same function on Python, but I realized that it is not
necessary. Python is already strict *by default*.

Python can help developers to warn them about some "corner cases". We
have already the -bb option for bytes/str warnings (in Python 3),
-Werror to convert warnings to exceptions, and ResourceWarning (since
Python 3.2) for unclosed files/sockets. I "just" would like a new
warning for an implicit locale encoding, so -Wcrossplatform would be as
easy as -Werror. -Werror is like Perl "use strict;" or PHP
error_reporting(E_ALL). Use -Wd if you prefer to display warnings
instead of raising exceptions.

See issue #11455 and #11470 for a new "CompatibilityWarning", it's not
"cross platform" but "cross Python" :-) It warns about implementation
details like non strings keys in a type dict.

Victor


From mal at egenix.com  Wed Jun 29 12:20:42 2011
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 29 Jun 2011 12:20:42 +0200
Subject: [Python-Dev] open(): set the default encoding to 'utf-8'
 in	Python 3.3?
In-Reply-To: <1309341057.8003.9.camel@marge>
References: <1309268585.21006.22.camel@marge>
	<4E09DF0C.90200@egenix.com>	<1309294298.23387.12.camel@marge>
	<4E0ADFF3.50100@egenix.com> <1309341057.8003.9.camel@marge>
Message-ID: <4E0AFC7A.502@egenix.com>

Victor Stinner wrote:
> Le mercredi 29 juin 2011 ? 10:18 +0200, M.-A. Lemburg a ?crit :
>> Victor Stinner wrote:
>>> Le mardi 28 juin 2011 ? 16:02 +0200, M.-A. Lemburg a ?crit :
>>>> How about a more radical change: have open() in Py3 default to
>>>> opening the file in binary mode, if no encoding is given (even
>>>> if the mode doesn't include 'b') ?
>>>
>>> I tried your suggested change: Python doesn't start.
>>
>> No surprise there: it's an incompatible change, but one that undoes
>> a wart introduced in the Py3 transition. Guessing encodings should
>> be avoided whenever possible.
> 
> It means that all programs written for Python 3.0, 3.1, 3.2 will stop
> working with the new 3.x version (let say 3.3). Users will have to
> migrate from Python 2 to Python 3.2, and then migration from Python 3.2
> to Python 3.3 :-(

I wasn't suggesting doing this for 3.3, but we may want to start
the usual feature change process to make the change eventually
happen.

> I would prefer a ResourceWarning (emited if the encoding is not
> specified), hidden by default: it doesn't break compatibility, and
> -Werror gives exactly the same behaviour that you expect.

ResourceWarning is the wrong type of warning for this. I'd
suggest to use a UnicodeWarning or perhaps create a new
EncodingWarning instead.

>> This demonstrates that Python's stdlib is still not being explicit
>> about the encoding issues. I suppose that things just happen to work
>> because we mostly use ASCII files for configuration and setup.
> 
> I did more tests. I found some mistakes and sometimes the binary mode
> can be used, but most function really expect the locale encoding (it is
> the correct encoding to read-write files). I agree that it would be to
> have an explicit encoding="locale", but make it mandatory is a little
> bit rude.

Again: Using a locale based default encoding will not work out
in the long run. We've had those discussions many times in the
past.

I don't think there's anything bad with having the user require
to set an encoding if he wants to read text. It makes him/her
think twice about the encoding issue, which is good.

And, of course, the stdlib should start using this
explicit-is-better-than-implicit approach as well.

>>> Then I tried my suggestion (use "utf-8" by default): Python starts
>>> correctly, I can build it (run "make") and... the full test suite pass
>>> without any change. (I'm testing on Linux, my locale encoding is UTF-8.)
>>
>> I bet it would also with "ascii" in most cases. Which then just
>> means that the Python build process and test suite is not a good
>> test case for choosing a default encoding.
>>
>> Linux is also a poor test candidate for this, since most user setups
>> will use UTF-8 as locale encoding. Windows, OTOH, uses all sorts of
>> code page encodings (usually not UTF-8), so you are likely to hit
>> the real problem cases a lot easier.
> 
> I also ran the test suite on my patched Python (open uses UTF-8 by
> default) with ASCII locale encoding (LANG=C), the test suite does also
> pass. Many tests uses non-ASCII characters, some of them are skipped if
> the locale encoding is unable to encode the tested text.

Thanks for checking. So the build process and test suite are
indeed not suitable test cases for the problem at hand. With
just ASCII files to decode, Python will simply never fail
to decode the content, regardless of whether you use an ASCII,
UTF-8 or some Windows code page as locale encoding.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 29 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ncoghlan at gmail.com  Wed Jun 29 14:37:00 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 29 Jun 2011 22:37:00 +1000
Subject: [Python-Dev] [Python-checkins] cpython: Issue #12400: fix
 test_faulthandler if regrtest captures sys.stderr
In-Reply-To: <E1QbtBM-0000Px-G1@dinsdale.python.org>
References: <E1QbtBM-0000Px-G1@dinsdale.python.org>
Message-ID: <BANLkTinKHCopQ9yXFMe=gQ==Jk1S0gYPpQ@mail.gmail.com>

On Wed, Jun 29, 2011 at 9:43 PM, victor.stinner
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/80c7d63325f6
> changeset: ? 71065:80c7d63325f6
> user: ? ? ? ?Victor Stinner <victor.stinner at haypocalc.com>
> date: ? ? ? ?Wed Jun 29 13:44:05 2011 +0200
> summary:
> ?Issue #12400: fix test_faulthandler if regrtest captures sys.stderr
>
> faulthandler.enable() requires that sys.stderr has a fileno() method.

Would there be any value in falling back to sys.__stderr__ in the case
where sys.stderr has been redirected?

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Wed Jun 29 19:00:13 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 29 Jun 2011 17:00:13 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
Message-ID: <loom.20110629T185646-901@post.gmane.org>

PEP 397 (Python launcher for Windows) has a reference implementation in Python.
Does anyone know of a C implementation, or is planning/working on one? I realise
this is the final objective, so such implementation might be premature, but
perhaps someone has been experimenting ...

Regards,

Vinay Sajip



From victor.stinner at haypocalc.com  Wed Jun 29 20:05:27 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 29 Jun 2011 20:05:27 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.2): Issue #12400:
 runtest() truncates the StringIO stream before a new test
In-Reply-To: <4E0B5B49.7010905@udel.edu>
References: <E1QbwiZ-0006mn-Ha@dinsdale.python.org> <4E0B5B49.7010905@udel.edu>
Message-ID: <1309370727.7968.3.camel@marge>

Le mercredi 29 juin 2011 ? 13:05 -0400, Terry Reedy a ?crit :
> On 6/29/2011 11:30 AM, victor.stinner wrote:
> 
> > summary:
> >    Issue #12400: runtest() truncates the StringIO stream before a new test
> >
> > files:
> >    Lib/test/regrtest.py |  1 +
> >    1 files changed, 1 insertions(+), 0 deletions(-)
> >
> >
> > diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py
> > --- a/Lib/test/regrtest.py
> > +++ b/Lib/test/regrtest.py
> > @@ -793,6 +793,7 @@
> >               if runtest.stringio is None:
> >                   runtest.stringio = io.StringIO()
> >               stream = runtest.stringio
> > +            stream.truncate(0)
> 
> You *MUST* seek to 0 to reset the file position, which I presume is your 
> intention. 'Resize' does not mean what you probably think is does (and 
> what I thought once either ;-).
> 
> "truncate(size=None)
> Resize the stream to the given size in bytes (or the current position if 
> size is not specified). The current stream position isn?t changed."
> 
>  >>> s=io.StringIO()
>  >>> s.write('abc')
> 3
>  >>> s.truncate(0)
> 0
>  >>> s.tell()
> 3
>  >>> s.write('abc')
> 3
>  >>> s.getvalue()
> '\x00\x00\x00abc'

Oh crap. I read _pyio source code and .truncate(0) was looking for the
right method to "reset" a StringIO. I tried .truncate(0) in a terminal,
but the zeros ('\x00') are "hidden". sys.stdout.write("\0") doesn't
print anything in my Linux terminal.

Fixed by commit 450209efe272, thank you.

Victor


From martin at v.loewis.de  Wed Jun 29 23:38:48 2011
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 29 Jun 2011 23:38:48 +0200
Subject: [Python-Dev] svn.python.org confusion
In-Reply-To: <nad-47439B.09372828062011@news.gmane.org>
References: <nad-47439B.09372828062011@news.gmane.org>
Message-ID: <4E0B9B68.6090207@v.loewis.de>


> Could some text be added to the svn browser pages to at least indicate 
> that the repo is no longer being updated with a link to the 
> hg.python.org browser?

Adding some text to only some pages (namely those that have been
migrated to mercurial) is difficult - so difficult that I won't
have time for it within the next 12 months. Contributions are welcome.

> I'm not sure what to do about the repos 
> themselves if people attempt to do an svn co.  Perhaps that should just 
> be disabled totally for python?

It would certainly be possible to "svn rm" the trunk directory, if that
is any help.

Of course, this would also break all URL links into this tree, of which
there are still plenty (and they likely won't go away in the years to
come).

So I would rather check a file into the directory with the name

THIS_SUBVERSION_REPOSITORY_HAS_BEEN_MIGRATED_TO_MERCURIAL_DO_NOT_USE_IT_FOR_ANYTHING_BUT_ARCHEOLOGICAL_RESEARCH.txt

See also

http://python.cvs.sourceforge.net/viewvc/python/python/
http://python.cvs.sourceforge.net/viewvc/python/python/dist/src/

Regards,
Martin

From ezio.melotti at gmail.com  Thu Jun 30 03:14:20 2011
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Thu, 30 Jun 2011 04:14:20 +0300
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Fixup repr for
	dict_proxy objects.
In-Reply-To: <E1Qc4Xh-0007fx-EO@dinsdale.python.org>
References: <E1Qc4Xh-0007fx-EO@dinsdale.python.org>
Message-ID: <BANLkTimr_aBNryYHzsCGYrxbeydmmTvpMw@mail.gmail.com>

Hi,

On Thu, Jun 30, 2011 at 2:51 AM, raymond.hettinger <
python-checkins at python.org> wrote:

> http://hg.python.org/cpython/rev/9a0b6c07b488
> changeset:   71086:9a0b6c07b488
> branch:      2.7
> parent:      71055:cdfcd00873cd
> user:        Raymond Hettinger <python at rcn.com>
> date:        Thu Jun 30 00:44:36 2011 +0100
> summary:
>  Fixup repr for dict_proxy objects.
>


This was already fixed in a slightly different way in 3.x.  Unless there's a
valid reason to have two different implementations of dict_proxy.__repr__ in
2.x and 3.x, I would suggest to pick the best one and use it on all the
branches.
See also http://bugs.python.org/issue5587

Best Regards,
Ezio Melotti
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110630/6389f308/attachment.html>

From jnoller at gmail.com  Thu Jun 30 06:05:32 2011
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 30 Jun 2011 00:05:32 -0400
Subject: [Python-Dev] speed.python.org machine online
Message-ID: <BANLkTik232OZYLCGMbrUoBr=Wg82084Oog@mail.gmail.com>

I've posted a more expansive entry on my blog:
http://jessenoller.com/2011/06/29/announcing-the-new-speed-python-org-machine/

But the short version, that as discussed at the VM and language
summit, we now have a hosted machine dedicated to the running of
cross-interpreter speed tests, etc. The hardware was generously donate
by HP and the hosting provided, again, free, by OSU/OSL.

DL380 HP DL380G7 X5670 LFF (2U)
Dual HP NC382i Dual Port Mul?ti?func?tion Giga?bit Server Adapters
HP Smart Array P410i/1GB FBWC Controller
4x 4GB (1x4GB) Dual Rank x4 PC3-10600 (DDR3-1333) Reg?is?tered CAS-9 Mem?ory Kit
2x HP 750W Com?mon Slot Gold Hot Plug Power Sup?ply Kit
HP iLO Advanced includ?ing 1yr 24x7 Tech?ni?cal Sup?port and Updates
Elec?tronic  License
4x HP 300GB 6G SAS 15K rpm LFF (3.5-inch) Dual Port Enter?prise 3yr
War?ranty Hard Drive
2   HP DL380 G7 Intel? Xeon? X5680 (3.33GHz/6-core/130W/12MB) FIO Proces?sor Kit

With hyperthreading on, the machine has 24 cores, and handily
translates pypy using cpython 2.7 in about half the time it typically
takes.

I am looking forward to handing this over to the team who will be
running with the project from here on out - special thanks to Van, Bob
Gobeille at HP and the entire OSU/OSL team.

jesse

From skippy.hammond at gmail.com  Thu Jun 30 06:23:13 2011
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Thu, 30 Jun 2011 14:23:13 +1000
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <loom.20110629T185646-901@post.gmane.org>
References: <loom.20110629T185646-901@post.gmane.org>
Message-ID: <4E0BFA31.8020200@gmail.com>

On 30/06/2011 3:00 AM, Vinay Sajip wrote:
> PEP 397 (Python launcher for Windows) has a reference implementation in Python.
> Does anyone know of a C implementation, or is planning/working on one? I realise
> this is the final objective, so such implementation might be premature, but
> perhaps someone has been experimenting ...

Not yet - my last update of the PEP has made the existing reference 
implementation out-of-date, so I want to work on that before starting on 
the C version.  However, seeing as my note about the most recent PEP 
update attracted zero comments, I admit I lost any sense of urgency on 
actually doing this...  I'll make an effort to update that reference 
impl in the next week or so (but obviously anyone else is free to help ;)

Cheers,

Mark

From ericsnowcurrently at gmail.com  Thu Jun 30 06:55:14 2011
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Wed, 29 Jun 2011 22:55:14 -0600
Subject: [Python-Dev] speed.python.org machine online
In-Reply-To: <BANLkTik232OZYLCGMbrUoBr=Wg82084Oog@mail.gmail.com>
References: <BANLkTik232OZYLCGMbrUoBr=Wg82084Oog@mail.gmail.com>
Message-ID: <BANLkTi=THEsg9P+bSvVUHQpHbFkDAT9zaw@mail.gmail.com>

On Wed, Jun 29, 2011 at 10:05 PM, Jesse Noller <jnoller at gmail.com> wrote:
> I've posted a more expansive entry on my blog:
> http://jessenoller.com/2011/06/29/announcing-the-new-speed-python-org-machine/
>
> But the short version, that as discussed at the VM and language
> summit, we now have a hosted machine dedicated to the running of
> cross-interpreter speed tests, etc. The hardware was generously donate
> by HP and the hosting provided, again, free, by OSU/OSL.
>
> DL380 HP DL380G7 X5670 LFF (2U)
> Dual HP NC382i Dual Port Mul?ti?func?tion Giga?bit Server Adapters
> HP Smart Array P410i/1GB FBWC Controller
> 4x 4GB (1x4GB) Dual Rank x4 PC3-10600 (DDR3-1333) Reg?is?tered CAS-9 Mem?ory Kit
> 2x HP 750W Com?mon Slot Gold Hot Plug Power Sup?ply Kit
> HP iLO Advanced includ?ing 1yr 24x7 Tech?ni?cal Sup?port and Updates
> Elec?tronic ?License
> 4x HP 300GB 6G SAS 15K rpm LFF (3.5-inch) Dual Port Enter?prise 3yr
> War?ranty Hard Drive
> 2 ? HP DL380 G7 Intel? Xeon? X5680 (3.33GHz/6-core/130W/12MB) FIO Proces?sor Kit
>
> With hyperthreading on, the machine has 24 cores, and handily
> translates pypy using cpython 2.7 in about half the time it typically
> takes.
>
> I am looking forward to handing this over to the team who will be
> running with the project from here on out - special thanks to Van, Bob
> Gobeille at HP and the entire OSU/OSL team.
>

Thanks for the continuing effort, Jesse, Van, and everyone!  This is
why Python is great and not just good.  I can't say thank you enough
to all the people that work so hard on tracker issues, documentation,
conferences, outreach, infrastructure, and all the pieces that make
Python more than just a good idea.  Not to discount the effort leading
up to the ideas, but the Python community realizes them through the
hard work of so many individuals willing to "sacrifice" for something
they love.

I've been following this list for a year and have not seen much
recognition, so it likely goes without saying (or I've just broken an
unwritten rule :), but sometimes it needs to be said out loud
regardless and sometimes loudly.  So thanks everyone!

-eric

> jesse
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>

From raymond.hettinger at gmail.com  Thu Jun 30 08:41:57 2011
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Thu, 30 Jun 2011 07:41:57 +0100
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Fixup repr for
	dict_proxy objects.
In-Reply-To: <BANLkTimr_aBNryYHzsCGYrxbeydmmTvpMw@mail.gmail.com>
References: <E1Qc4Xh-0007fx-EO@dinsdale.python.org>
	<BANLkTimr_aBNryYHzsCGYrxbeydmmTvpMw@mail.gmail.com>
Message-ID: <C5CE8E3F-9BF5-4B72-9CA7-CAC16E834097@gmail.com>


On Jun 30, 2011, at 2:14 AM, Ezio Melotti wrote:

> Hi,
> 
> On Thu, Jun 30, 2011 at 2:51 AM, raymond.hettinger <python-checkins at python.org> wrote:
>  Fixup repr for dict_proxy objects.
> 
> 
> This was already fixed in a slightly different way in 3.x. 

The %R formatting code is not available in 2.x

Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110630/9c18f7d3/attachment.html>

From mail at timgolden.me.uk  Thu Jun 30 09:34:19 2011
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 30 Jun 2011 08:34:19 +0100
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <4E0BFA31.8020200@gmail.com>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
Message-ID: <4E0C26FB.9040704@timgolden.me.uk>

On 30/06/2011 05:23, Mark Hammond wrote:
> On 30/06/2011 3:00 AM, Vinay Sajip wrote:
>> PEP 397 (Python launcher for Windows) has a reference implementation
>> in Python.
>> Does anyone know of a C implementation, or is planning/working on one?
>> I realise
>> this is the final objective, so such implementation might be
>> premature, but
>> perhaps someone has been experimenting ...
>
> Not yet - my last update of the PEP has made the existing reference
> implementation out-of-date, so I want to work on that before starting on
> the C version. However, seeing as my note about the most recent PEP
> update attracted zero comments..

Sorry, didn't realise you couldn't see me silently nodding my
head from the other side of the world ;) I really must work
on those communication skills...

TJG

From ncoghlan at gmail.com  Thu Jun 30 12:58:52 2011
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 30 Jun 2011 20:58:52 +1000
Subject: [Python-Dev] [Python-checkins] cpython: store the current scope
 on the stack right away
In-Reply-To: <E1Qc8KC-0006YH-Tw@dinsdale.python.org>
References: <E1Qc8KC-0006YH-Tw@dinsdale.python.org>
Message-ID: <BANLkTi=Bsn5MH_HSCK+Q7EGyPyXuxE+Q2Q@mail.gmail.com>

On Thu, Jun 30, 2011 at 1:53 PM, benjamin.peterson
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/151142c0c5b1
> changeset: ? 71087:151142c0c5b1
> parent: ? ? ?71084:9aa3fcab76f0
> user: ? ? ? ?Benjamin Peterson <benjamin at python.org>
> date: ? ? ? ?Wed Jun 29 22:52:39 2011 -0500
> summary:
> ?store the current scope on the stack right away

Heh, I was going to comment that the spate of refleaks after your
previous commit looked mighty suspicious :)

End result is simpler, cleaner code overall, though.

Cheers,
Nick.

-- 
Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia

From fuzzyman at voidspace.org.uk  Thu Jun 30 13:13:25 2011
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 30 Jun 2011 12:13:25 +0100
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
 implementation
In-Reply-To: <4E0C26FB.9040704@timgolden.me.uk>
References: <loom.20110629T185646-901@post.gmane.org>	<4E0BFA31.8020200@gmail.com>
	<4E0C26FB.9040704@timgolden.me.uk>
Message-ID: <4E0C5A55.6000706@voidspace.org.uk>

On 30/06/2011 08:34, Tim Golden wrote:
> On 30/06/2011 05:23, Mark Hammond wrote:
>> On 30/06/2011 3:00 AM, Vinay Sajip wrote:
>>> PEP 397 (Python launcher for Windows) has a reference implementation
>>> in Python.
>>> Does anyone know of a C implementation, or is planning/working on one?
>>> I realise
>>> this is the final objective, so such implementation might be
>>> premature, but
>>> perhaps someone has been experimenting ...
>>
>> Not yet - my last update of the PEP has made the existing reference
>> implementation out-of-date, so I want to work on that before starting on
>> the C version. However, seeing as my note about the most recent PEP
>> update attracted zero comments..
>
> Sorry, didn't realise you couldn't see me silently nodding my
> head from the other side of the world ;) I really must work
> on those communication skills...

I have that email (the update one from Mark not the silent nodding from 
Tim) still sitting in my inbox waiting for me to properly read through 
and comment on... Sorry for being useless, I'll try and move it up the 
priority list.

I really like the PEP and think it will be a *huge* step forward for 
Windows users - so it's purely the details that need thrashing out (heh).

In the latest update Mark also addressed my main concern, making the 
launcher configurable so it can also be used by alternative 
implementations (particularly IronPython for Windows). I've copied Jeff 
Hardy and Dino (IronPython maintainers) in on this, in the hope that 
they might take a look at the latest version of the PEP too...

All the best,

Michael

>
> TJG
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html


From vinay_sajip at yahoo.co.uk  Thu Jun 30 14:09:43 2011
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 30 Jun 2011 12:09:43 +0000 (UTC)
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com>
Message-ID: <loom.20110630T140918-870@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:

> Not yet - my last update of the PEP has made the existing reference 
> implementation out-of-date, so I want to work on that before starting on 
> the C version.  However, seeing as my note about the most recent PEP 
> update attracted zero comments, I admit I lost any sense of urgency on 
> actually doing this...  I'll make an effort to update that reference 
> impl in the next week or so (but obviously anyone else is free to help ;)

Sorry, I just recently came across the PEP when Tim Golden pointed it out to
me, when I asked a question about executable script support in packaging. I
think I was probably not so focused on Windows at the time of the original
announcement ...

There's a lot to like in the PEP, and I have some questions relating to the
latest version:

1. In the section on shebang line parsing, it says "If the command starts with
the definition of a customized command followed by a space character, the
customized command will be used." Sorry if I'm being dense, but what's the
significance of the trailing space character? In fact, your vpython example in
the customeised comments section doesn't show a trailing space - you've used
'#! vpython' as the shebang line.

2. The section on .ini files says that "Commands specified in the [.ini file
in the] "application directory" [APPDATA] will have precedence over the one
next to the [launcher] executable." What's the benefit of this? If you have
only one launcher executable of one type (say console, 32-bit) on a system,
then there's no point in having two locations for .ini files. If you have
multiple copies of the launcher and multiple .ini files next to them, then
with this precedence order, you can't specialise the behaviour of anything in
a specific launcher .ini that's also specified in the APPDATA .ini. Doesn't it
make more sense to look for a setting in any file next to the launcher, and if
not found there, look in the APPDATA? That way common things can be defined in
the APPDATA .ini and overridden for special cases in the launcher-adjacent
.ini. Or have I got completely the wrong end of the stick?

By the way, I've already converted the existing Python reference implementation
to C (I did it before I saw your response to my post). It basically works in
that it does what the Python version does, but doesn't include any handling of
-32 suffixes or .ini files. I can post this on BitBucket if anyone's interested,
but it's a work in progress. I'm working on some simple tests.

Regards,

Vinay Sajip



From p.f.moore at gmail.com  Thu Jun 30 14:50:11 2011
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 30 Jun 2011 13:50:11 +0100
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
In-Reply-To: <4E0C5A55.6000706@voidspace.org.uk>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com> <4E0C26FB.9040704@timgolden.me.uk>
	<4E0C5A55.6000706@voidspace.org.uk>
Message-ID: <BANLkTinjAwbTC0fvC+-+VkVozfbn-_z-5A@mail.gmail.com>

On 30 June 2011 12:13, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> I have that email (the update one from Mark not the silent nodding from Tim)
> still sitting in my inbox waiting for me to properly read through and
> comment on... Sorry for being useless, I'll try and move it up the priority
> list.
>
> I really like the PEP and think it will be a *huge* step forward for Windows
> users - so it's purely the details that need thrashing out (heh).

That's my situation, too. I'll try to look through it properly in the
next day or two.

Paul.

From merwok at netwok.org  Thu Jun 30 16:17:32 2011
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 30 Jun 2011 16:17:32 +0200
Subject: [Python-Dev] svn.python.org confusion
In-Reply-To: <4E0B9B68.6090207@v.loewis.de>
References: <nad-47439B.09372828062011@news.gmane.org>
	<4E0B9B68.6090207@v.loewis.de>
Message-ID: <4E0C857C.6040906@netwok.org>

Adding a THIS_REPOSITORY_HAS_MOVED file to recent branches looks good.

Regards

From barry at python.org  Thu Jun 30 16:32:51 2011
From: barry at python.org (Barry Warsaw)
Date: Thu, 30 Jun 2011 15:32:51 +0100
Subject: [Python-Dev] svn.python.org confusion
In-Reply-To: <4E0C857C.6040906@netwok.org>
References: <nad-47439B.09372828062011@news.gmane.org>
	<4E0B9B68.6090207@v.loewis.de> <4E0C857C.6040906@netwok.org>
Message-ID: <20110630153251.751dddfd@snowdog>

On Jun 30, 2011, at 04:17 PM, ?ric Araujo wrote:

>Adding a THIS_REPOSITORY_HAS_MOVED file to recent branches looks good.

I'm not against adding this to svn, but please be sure these files don't leak
into the tarballs should we need to cut another 2.5 or 2.6 release.  I think
that just means tweaking sandbox/release/release.py.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110630/ac774654/attachment.pgp>

From tseaver at palladion.com  Thu Jun 30 17:40:30 2011
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 30 Jun 2011 11:40:30 -0400
Subject: [Python-Dev] svn.python.org confusion
In-Reply-To: <20110630153251.751dddfd@snowdog>
References: <nad-47439B.09372828062011@news.gmane.org>	<4E0B9B68.6090207@v.loewis.de>
	<4E0C857C.6040906@netwok.org> <20110630153251.751dddfd@snowdog>
Message-ID: <iui5dd$bik$1@dough.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/30/2011 10:32 AM, Barry Warsaw wrote:
> On Jun 30, 2011, at 04:17 PM, ?ric Araujo wrote:
> 
>> Adding a THIS_REPOSITORY_HAS_MOVED file to recent branches looks good.
> 
> I'm not against adding this to svn, but please be sure these files don't leak
> into the tarballs should we need to cut another 2.5 or 2.6 release.  I think
> that just means tweaking sandbox/release/release.py.

The fact that releases might / will still be made from those branches
argues against including the file on them at all:  they are in fact the
"canonical" repository locations, even if most of the work is done in hg
and patched into them.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4MmO4ACgkQ+gerLs4ltQ7ybwCgqIkXCdYWIdPM9dHFDa658nLe
RYkAnRccCcuCjxQvYSn/jseLJU/ZTmcQ
=+fEZ
-----END PGP SIGNATURE-----


From jdhardy at gmail.com  Thu Jun 30 19:31:34 2011
From: jdhardy at gmail.com (Jeff Hardy)
Date: Thu, 30 Jun 2011 10:31:34 -0700
Subject: [Python-Dev] PEP 397 (Python launcher for Windows) reference
	implementation
In-Reply-To: <4E0C5A55.6000706@voidspace.org.uk>
References: <loom.20110629T185646-901@post.gmane.org>
	<4E0BFA31.8020200@gmail.com> <4E0C26FB.9040704@timgolden.me.uk>
	<4E0C5A55.6000706@voidspace.org.uk>
Message-ID: <BANLkTi=c53Fm_o9BpYiWf8451e6CVhB=Qw@mail.gmail.com>

On Thu, Jun 30, 2011 at 4:13 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> In the latest update Mark also addressed my main concern, making the
> launcher configurable so it can also be used by alternative implementations
> (particularly IronPython for Windows). I've copied Jeff Hardy and Dino
> (IronPython maintainers) in on this, in the hope that they might take a look
> at the latest version of the PEP too...

I skimmed it a while back but haven't had the time to give an in-depth
study. On the surface it seems fine, but the devil is in the
details...

- Jeff

From ulrich.eckhardt at dominolaser.com  Thu Jun 30 21:13:46 2011
From: ulrich.eckhardt at dominolaser.com (Ulrich Eckhardt)
Date: Thu, 30 Jun 2011 21:13:46 +0200
Subject: [Python-Dev] time.sleep(-1) behaviour
Message-ID: <201106302113.47144.ulrich.eckhardt@dominolaser.com>

Hi!

This is a request for clarification for the thread "how to call a function for 
evry 10 seconds" from the user mailinglist/newsgroup.


The gist is this:
1. On Linux/Python 2.6, time.sleep(-1.0) raises an IOError.
2. On MS Windows/Python 2.5 or 2.7 this sleeps forever. It seems that 
converting this to a 32-bit integer for Sleep() causes an underflow.


Questions:
1. Is the IOError somehow explainable?
2. Is the IOError acceptable or a bug?
3. Is the behaviour under MS Windows acceptable or a bug?
4. Since time.sleep() is documented to possibly sleep a bit longer than 
specified, someone suggested that the overall sleep time could also be 
truncated to a minimum of zero, i.e. treating negative values as zero.


Greetings!

Uli
**************************************************************************************
Domino Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland
Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at http://www.dominolaser.com
**************************************************************************************
Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden.
E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Domino Laser GmbH ist f?r diese Folgen nicht verantwortlich.
**************************************************************************************


From rdmurray at bitdance.com  Thu Jun 30 21:21:44 2011
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 30 Jun 2011 15:21:44 -0400
Subject: [Python-Dev] time.sleep(-1) behaviour
In-Reply-To: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
References: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
Message-ID: <20110630192145.7C83D2506C1@webabinitio.net>

On Thu, 30 Jun 2011 21:13:46 +0200, Ulrich Eckhardt <ulrich.eckhardt at dominolaser.com> wrote:
> 1. Is the IOError somehow explainable?
> 2. Is the IOError acceptable or a bug?
> 3. Is the behaviour under MS Windows acceptable or a bug?
> 4. Since time.sleep() is documented to possibly sleep a bit longer than 
> specified, someone suggested that the overall sleep time could also be 
> truncated to a minimum of zero, i.e. treating negative values as zero.

Please file a bug report at bugs.python.org.

--
R. David Murray           http://www.bitdance.com

From victor.stinner at haypocalc.com  Thu Jun 30 22:11:42 2011
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 30 Jun 2011 22:11:42 +0200
Subject: [Python-Dev] time.sleep(-1) behaviour
In-Reply-To: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
References: <201106302113.47144.ulrich.eckhardt@dominolaser.com>
Message-ID: <1309464702.29934.2.camel@marge>

Le jeudi 30 juin 2011 ? 21:13 +0200, Ulrich Eckhardt a ?crit :
> Hi!
> 
> This is a request for clarification for the thread "how to call a function for 
> evry 10 seconds" from the user mailinglist/newsgroup.
> 
> 
> The gist is this:
> 1. On Linux/Python 2.6, time.sleep(-1.0) raises an IOError.
> 2. On MS Windows/Python 2.5 or 2.7 this sleeps forever. It seems that 
> converting this to a 32-bit integer for Sleep() causes an underflow.

select.select() was changed recently to raise an exception if the
timeout is negative to get the same behaviour on all platforms.

signal.sigtimedwait() (recently added function) raises also an exception
if the timeout is negative.

If it's an overflow, Python should raise an OverflowError, or raise
another error, but not sleep for 2^32-1 (or 2^64-1 ?) seconds.

Victor