Mailman 3 November 2009 - Python-Dev

People want CPAN :-)
by ssteinerX＠gmail.com 10 Nov '09

10 Nov '09

On Nov 7, 2009, at 3:20 AM, Ben Finney wrote: > Guido van Rossum <guido(a)python.org> writes: > >> On Fri, Nov 6, 2009 at 2:52 PM, David Lyon >> <david.lyon(a)preisshare.net> wrote: >>> > I think buildbot-style test runs for PyPI packages would raise average > package quality on PyPI. Please excuse the cross-post but I wanted to make sure that all these "CPAN for Python" discussions got this message and I've lost track of which list which part of what discussion had occurred on. We are currently extending our distutils/Distribute test system to include installation of a broad range of packages as part of the pre- release process for a future release of Distribute and as part of our "smoke" test for distutils/Distribute. Eventually, the goal is to integrate this with our buildbot system but that's a ways off. Our goal is to install a range of packages and, where practicable, actually run and record any errors with the packages' individual test suites. Right now, our "smoke" test only does Twisted and numpy. We've discussed how to collect test results from Twisted trial and we'll be working on similar things for other test runners (nose et al.). For Twisted, we're going to install and test both the current release version and an svn checkout from trunk. It would be an extension of that concept to install and test *all* packages from PyPI but would, obviously, take considerable horsepower (and time) to run such an exhaustive test (especially if we're talking about 2.4?, 2.5, 2.6, 2.7, and 3.1+. Right now I'm extending the configuration file for our smoke test to allow for various test runners (e.g. nose, twisted trial, etc.) so we can "smoke out" more installation problems and/or failed tests after installation. For the first pass, I'm just focusing on Twisted and trial, then numpy, then finding packages that support nose so that I can collect the data on what ran, what passed, and what didn't. I'm planning on collecting this all in a database and making some simple API so that it can be mined by very simple apps later. At the point where that infrastructure is in place, we could pretty easily mine the data to do all kinds of crazy things people have mentioned like: * A ranking system of test coverage * Complexity analysis * Test coverage * Run pylint, pyflakes, 2to3, whatever automated measurement tools over the code * Send test failure messages to maintainers (maybe with opt-in in the new meta-data). * Whatever! We're actively working on this right now; anyone who wants to lend a hand is welcome to contact me off-list and we can talk about what types of things we are needing and where we could use a hand. All in all, I think this could be a big leap forward for the Python distribution ecosystem whether or not we eventually write the PyPan I wished for as a new Perl refugee. Thanks, S

3 3

Python 2.6: OverflowError: signed integer is greater than maximum
by Jasper Lievisse Adriaanse 09 Nov '09

09 Nov '09

Hi, while trying to get Python 2.6 working on OpenBSD/sgi (64-bit port) I ran into the following during build: OverflowError: signed integer is greater than maximum I ran the command that triggered this by hand with -v added: (sgi Python-2.6.3 40)$ export PATH; PATH="`pwd`:$PATH"; export PYTHONPATH; PYTHONPATH="`pwd`/Lib"; export DYLD_FRAMEWORK_PATH; DYLD_FRAMEWORK_PATH="`pwd`"; export EXE; EXE=""; cd ./Lib/plat-openbsd4; ./regen python$EXE -v ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h # installing zipimport hook import zipimport # builtin # installed zipimport hook # /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.pyc matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.py import site # precompiled from /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.pyc 'import site' failed; traceback: Traceback (most recent call last): File "/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.py", line 61, in <module> import sys OverflowError: signed integer is greater than maximum import encodings # directory /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings # /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.pyc matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.py import encodings # precompiled from /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.pyc Python 2.6.3 (r263:75183, Nov 6 2009, 09:50:33) [GCC 3.3.5 (propolice)] on openbsd4 Type "help", "copyright", "credits" or "license" for more information. # /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.pyc matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.py import re # precompiled from /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.pyc Traceback (most recent call last): File "../../Tools/scripts/h2py.py", line 24, in <module> import sys, re, getopt, os File "/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.py", line 104, in <module> import sys OverflowError: signed integer is greater than maximum # clear __builtin__._ # clear sys.path # clear sys.argv # clear sys.ps1 # clear sys.ps2 # clear sys.exitfunc # clear sys.exc_type # clear sys.exc_value # clear sys.exc_traceback # clear sys.last_type # clear sys.last_value # clear sys.last_traceback # clear sys.path_hooks # clear sys.path_importer_cache # clear sys.meta_path # clear sys.flags # clear sys.float_info # restore sys.stdin # restore sys.stdout # restore sys.stderr # cleanup __main__ # cleanup[1] zipimport # cleanup[1] signal # cleanup[1] exceptions # cleanup[1] _warnings # cleanup sys # cleanup __builtin__ # cleanup ints: 3 unfreed ints # cleanup floats (sgi plat-openbsd4 41)$ Does anyone know what's going on here, or how to fix it? I can provide a full build and configure log if needed. Cheers, Jasper -- "Intelligence should guide our actions, but in harmony with the texture of the situation at hand" -- Francisco Varela

3 3

PEP 3003 - Python Language Moratorium
by Guido van Rossum 09 Nov '09

09 Nov '09

I've checked draft (!) PEP 3003, "Python Language Moratorium", into SVN. As authors I've listed Jesse, Brett and myself. On python-ideas the moratorium idea got fairly positive responses (more positive than I'd expected, in fact) but I'm bracing myself for fierce discussion here on python-dev. It's important to me that if if this is accepted it is a "rough consensus" decision (working code we already have plenty of :-), not something enforced by a vocal minority or an influential individual such as myself. If there's too much opposition I'll withdraw the PEP so as not to waste everybody's time with a fruitless discussion. The PEP tries to spell out some gray areas but I'm sure there will be others; that's life. Do note that the PEP proposes to be *retroactive* back to the 3.1 release, i.e. the "frozen" version of the language is the state in which it was released as 3.1. -- --Guido van Rossum (python.org/~guido)

42 100

Updates to XP-4 and FreeBSD build slaves
by David Bolen 09 Nov '09

09 Nov '09

I think in a prior discussion, it was suggested that build slave updates were ok for this list - I apologize to those who may not be interested. I've just completed some updates to my two build slaves. XP-4 is now running XP Pro SP3 (was SP2) and is using the full version of VS 2008 (was previously using Express). FreeBSD is now running FreeBSD 6.4-RELEASE (was 6.2-RELEASE). Both are new VMs, and also now use the latest buildbot (0.7.11p3) release. For at least the next few days they are being run on a different host while I do some long-awaited maintenance to the old host. I've done manual builds on both, as well as reissuing a few builds through the master, but if anyone notices anything strange please feel free to drop me a note. -- David PS: I did discover one interesting bug by starting fresh on Windows. It turns out that one of the distutils tests assumes Notepad entries in the registry that aren't actually present unless you run Notepad at least once. I hadn't needed to run it, but then noticed the build failures so ran it just to get the registry populated.

1 0

Re: [Python-Dev] PEP 3003 - Python Language Moratorium
by exarkun＠twistedmatrix.com 07 Nov '09

07 Nov '09

On 01:50 pm, dripton(a)ripton.net wrote: >On 2009.11.06 08:25:29 +0100, Hagen F�rstenau wrote: >> >> A switch to enable deprecation warnings would give developers a >> >> chance to see them when migrating to a new version of python. And >>it >> >> would prevent users from having to deal with them. >> > >> > PendingDeprecationWarning is silent by default and requires a switch >>to >> > be enabled. >> >>So what we need is perhaps not a stdlib moratorium, but rather strict >>adherence to a sequence of PendingDeprecationWarning -> >>DeprecationWarning -> Removal? (The first one seems to have been >>underused so far.) > >+1 > >I really don't like the loud warnings for sha and md5 in 2.6, >considering that hashlib wasn't even added until 2.5. It should be >possible to write code that works across all the versions of Python 2 >that are currently shipping with mainstream Linux distrobutions without >deprecation warnings or conditional version-dependent code. > >To go a step further, maybe follow javac's example and emit just one >summary deprecation warning per program by default. Something >like "this program uses deprecated API calls. Run it with >python -deprecation to see the details." End users don't even need to be told this much, though (and if it's javac that does this for Java, then indeed Java end users probably aren't seeing this either). I think it would be great for deprecation warnings to be completely silent for a while *and* for the policy to be prominently documented so that developers, the people who really need this information, know how to get it. Jean-Paul

4 5

Summary of Python tracker Issues
by Python tracker 06 Nov '09

06 Nov '09

ACTIVITY SUMMARY (10/30/09 - 11/06/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2479 open (+15) / 16616 closed (+15) / 19095 total (+30) Open issues with patches: 991 Average duration of open issues: 683 days. Median duration of open issues: 437 days. Open Issues Breakdown open 2444 (+15) pending 34 ( +0) Issues Created Or Reopened (31) _______________________________ mac binary download link for 2.6.4 broken 10/30/09 CLOSED http://bugs.python.org/issue7243 reopened beard itertools.zip_longest behaves strangely with an iterable class 10/31/09 CLOSED http://bugs.python.org/issue7244 created durban patch better Ctrl-C support in pdb (program can be resumed) 10/31/09 http://bugs.python.org/issue7245 created isandler getpass raises IOError when several returns are in stdin before 11/01/09 CLOSED http://bugs.python.org/issue7246 created avihu test_fcntl_64_bit from test_fcntl.py fails in Python 2.6.4 11/01/09 http://bugs.python.org/issue7247 created Chainsaw test_importlib uses a fixed name in /tmp 11/01/09 CLOSED http://bugs.python.org/issue7248 created pitrou Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6 11/02/09 CLOSED http://bugs.python.org/issue7249 created ryles wsgiref.handlers.CGIHandler caches os.environ, leaking info betw 11/02/09 http://bugs.python.org/issue7250 created snprbob86 patch, easy, 26backport Mark expected failures of test_math, test_cmath and test_round a 11/02/09 http://bugs.python.org/issue7251 created mark.dickinson buildbot list().index() should provide better error reporting 11/02/09 CLOSED http://bugs.python.org/issue7252 created djc AssertionError Tree is insane with *args and reduce 11/02/09 CLOSED http://bugs.python.org/issue7253 created jaraco Class members not properly initialized if declared outside of fu 11/02/09 CLOSED http://bugs.python.org/issue7254 created vladc6 "Default" word boundaries for Unicode data? 11/03/09 http://bugs.python.org/issue7255 created RegEx4All parse_qs and parse_qsl in urlparse are not documented as New in 11/03/09 CLOSED http://bugs.python.org/issue7256 created VeXocide Improve documentation of list.sort and sorted() 11/03/09 http://bugs.python.org/issue7257 created olau test_multiprocessing leaves a temporary directory 11/04/09 CLOSED http://bugs.python.org/issue7258 created pitrou Documentation update for operator module 11/04/09 CLOSED http://bugs.python.org/issue7259 created gagenellina patch SyntaxError with a not-existing offset for unicode code 11/04/09 CLOSED http://bugs.python.org/issue7260 created noam Document 2.x -> 3.x round changes in "What's New" documents. 11/04/09 http://bugs.python.org/issue7261 created mark.dickinson codecs.open() + eol (windows) 11/04/09 http://bugs.python.org/issue7262 created shamilbi Fix set.intersection docstring 11/04/09 http://bugs.python.org/issue7263 created belopolsky patch test_threading_local sometimes hangs when run with -R 11/04/09 CLOSED http://bugs.python.org/issue7264 created pitrou patch IDLE not working 11/05/09 CLOSED http://bugs.python.org/issue7265 created Carey test_lib2to3 failure under Windows 11/05/09 http://bugs.python.org/issue7266 created pitrou easy format method: c presentation type broken 11/05/09 http://bugs.python.org/issue7267 created doerwalter 2to3 execfile conversion changes behavior 11/06/09 CLOSED http://bugs.python.org/issue7268 created flashk Windows buildbot occasional DBFileExistsError failures in test_b 11/06/09 http://bugs.python.org/issue7269 created r.david.murray buildbot Tests for synchronization primitives 11/06/09 http://bugs.python.org/issue7270 created pitrou patch Typo in turtle module documentation. 11/06/09 http://bugs.python.org/issue7271 created mark.dickinson test_multiprocessing fails consistently with 'signal 12' on Free 11/06/09 http://bugs.python.org/issue7272 created mark.dickinson buildbot struct.pack adding extra '\x00' character in very specific case 11/06/09 CLOSED http://bugs.python.org/issue7273 created cmadrigal Issues Now Closed (37) ______________________ Minor Change For Better cross compile 788 days http://bugs.python.org/issue1115 gregory.p.smith patch IDLE fails to launch 669 days http://bugs.python.org/issue1743 richjtd IDLE won't start in custom directory. 414 days http://bugs.python.org/issue3881 srid patch Add "key" argument to "bisect" module functions 348 days http://bugs.python.org/issue4356 rhettinger patch xml.etree does not escape CR, LF and TAB characters within attri 110 days http://bugs.python.org/issue6492 ezio.melotti Compilation error if configuref --with-computed-gotos 93 days http://bugs.python.org/issue6603 mark.dickinson patch, easy fnmatch fails on filenames containing \n character 86 days http://bugs.python.org/issue6665 gregory.p.smith Intermittent failures in test_mailbox 51 days http://bugs.python.org/issue6896 pitrou patch test_signal fails on os x 10.6 29 days http://bugs.python.org/issue7042 mark.dickinson patch SysLogHandler can't handle Unicode 29 days http://bugs.python.org/issue7077 vinay.sajip Value error 'path is on drive c: start on drive d:' in os.path.r 8 days http://bugs.python.org/issue7195 ncoghlan Getpass echo's password to screen on 2.6, but not on 2.5 or 3.1 6 days http://bugs.python.org/issue7208 gregory.p.smith patch Proposed Syntax Checks in Test Suite 5 days http://bugs.python.org/issue7210 ChuckRhode patch select module - kevent ident field 64 bit issue 9 days http://bugs.python.org/issue7211 pitrou patch IDLE Subprocess Startup Error {Solved} 8 days http://bugs.python.org/issue7217 amaury.forgeotdarc thread reaping is imperfect 3 days http://bugs.python.org/issue7222 pitrou patch One obvious way to do interning 9 days http://bugs.python.org/issue7224 rhettinger patch test_hotshot fails on solaris 2 days http://bugs.python.org/issue7230 pitrou subprocess.Popen.stdout.flush fails os OS-X 10.6.1 5 days http://bugs.python.org/issue7240 pjenvey mac binary download link for 2.6.4 broken 0 days http://bugs.python.org/issue7243 beard itertools.zip_longest behaves strangely with an iterable class 1 days http://bugs.python.org/issue7244 rhettinger patch getpass raises IOError when several returns are in stdin before 1 days http://bugs.python.org/issue7246 avihu test_importlib uses a fixed name in /tmp 3 days http://bugs.python.org/issue7248 brett.cannon Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6 1 days http://bugs.python.org/issue7249 pitrou list().index() should provide better error reporting 0 days http://bugs.python.org/issue7252 benjamin.peterson AssertionError Tree is insane with *args and reduce 0 days http://bugs.python.org/issue7253 benjamin.peterson Class members not properly initialized if declared outside of fu 0 days http://bugs.python.org/issue7254 r.david.murray parse_qs and parse_qsl in urlparse are not documented as New in 0 days http://bugs.python.org/issue7256 georg.brandl test_multiprocessing leaves a temporary directory 0 days http://bugs.python.org/issue7258 pitrou Documentation update for operator module 0 days http://bugs.python.org/issue7259 georg.brandl patch SyntaxError with a not-existing offset for unicode code 0 days http://bugs.python.org/issue7260 ezio.melotti test_threading_local sometimes hangs when run with -R 1 days http://bugs.python.org/issue7264 pitrou patch IDLE not working 1 days http://bugs.python.org/issue7265 loewis 2to3 execfile conversion changes behavior 0 days http://bugs.python.org/issue7268 flashk struct.pack adding extra '\x00' character in very specific case 0 days http://bugs.python.org/issue7273 mark.dickinson socket.inet_aton returns 8 bytes on LP64 platforms 265 days http://bugs.python.org/issue1008086 gregory.p.smith 64bit, easy Investigated ref leak report related to thread regrtest.py 866 days http://bugs.python.org/issue1739118 ocean-city patch Top Issues Most Discussed (10) ______________________________ 14 itertools.zip_longest behaves strangely with an iterable class 1 days closed http://bugs.python.org/issue7244 14 64 bit python fails on Windows 7 12 days open http://bugs.python.org/issue7206 14 Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1 1036 days open http://bugs.python.org/issue1628484 11 2to3 execfile conversion changes behavior 0 days closed http://bugs.python.org/issue7268 11 test_debuglevel from test_telnetlib.py fails 78 days open http://bugs.python.org/issue6748 10 Intermittent failures in test_mailbox 51 days closed http://bugs.python.org/issue6896 9 cross compile patch 1915 days open http://bugs.python.org/issue1006238 8 One obvious way to do interning 9 days closed http://bugs.python.org/issue7224 7 wsgiref.handlers.CGIHandler caches os.environ, leaking info bet 4 days open http://bugs.python.org/issue7250 7 Documentation add note about SystemRandom 30 days open http://bugs.python.org/issue7076

1 0

2.7 Release? 2.7 == last of the 2.x line?
by ssteinerX＠gmail.com 06 Nov '09

06 Nov '09

Not that anyone has asked yet, but here's my opinion on two issues that have been raised on the python-dev mailing list lately: +1 on 2.7 release with as much 3.0 "easy-port goo" as is practicable without delaying the product beyond the tentative schedule. Sooner would, of course, be better but I'm sure PEP 373 was produced after due consideration of all pertinent factors. +1 on 2.7 being the last of the 2.x series. Enough already! I, personally, haven't even written my first line of 3.x code, nor have I had any good reason to. If I saw the actual end of the line at 2.7, I would actually start looking for 3.x versions of my favorite tools and would be much more inclined to help push them along ASAP. Right now, so much that I use on a daily basis doesn't even have a 3.x roadmap, much less any sort of working implementation, that I don't see switching to 3.x ever unless the 2.x line ends, and soon! Just my one vote. S

37 130

Refactoring installation schemes
by Tarek Ziadé 06 Nov '09

06 Nov '09

Hello, Since the addition of PEP 370, (per-user site packages), site.py and distutils/command/install.py are *both* providing the various installation directories for Python, depending on the system and the Python version. We have also started to discuss lately in various Mailing Lists the addition of new schemes for IronPython and Jython, meaning that we might add some more in both places. I would like to suggest a simplification by adding a dedicated module to manage these installation schemes in one single place in the stdlib. This new independant module would be used by site.py and distutils and would also make it easier for third party code to work with these schemes. Of course this new module would be rather simple and not add any new import statement to avoid any overhead when Python starts and loads site.py Regards Tarek

9 17

No buildbot to test wide unicode?
by Amaury Forgeot d'Arc 05 Nov '09

05 Nov '09

Hello, It seems that there is no buildbot to test a wide unicode build. On http://www.python.org/dev/buildbot/3.x/ all outputs of the "configure" steps show this message:: checking what type to use for str... unsigned short which looks like a ucs2 build to me. Since wide unicode is the standard chosen by some Linux distributions, it would make sense to have at least one buildbot running with --with-wide-unicode (3.x) or --enable-unicode=ucs4 (2.x). -- Amaury Forgeot d'Arc

4 4

nonlocal keyword in 2.x?
by Mike Krell 05 Nov '09

05 Nov '09

Is there any possibility of backporting support for the nonlocal keyword into a 2.x release? I see it's not in 2.6, but I don't know if that was an intentional design choice or due to a lack of demand / round tuits. I'm also not sure if this would fall under the scope of the proposed moratorium on new language features (although my first impression was that it could be allowed since it already exists in python 3. One of my motivations for asking is a recent blog post by Fernando Perez of IPython fame that describes an interesting decorator-based idiom inspired by Apple's Grand Central Dispatch which would allow many interesting possibilities for expressing parallelization and other manipulations of execution context for blocks of python code. Unfortunately, using the technique to its fullest extent requires the nonlocal keyword. The blog post is here: https://cirl.berkeley.edu/fperez/py4science/decorators.html Mike

17 44

Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
by Larry Bugbee 05 Nov '09

05 Nov '09

This is my problem to solve... I work with a lot of [non-production] crypto where byte strings are a normal way of manipulating and storing keys and other artifacts. Under Python2 I have few/no Unicode issues. With Python3's default string type being Unicode, there are a lot of subtle ways where my codes go south. Like I say, it is my problem. I will convert to Py3, but not likely before summer. Besides 2to3, any other tools/suggestions would be welcome. (Perhaps there is a Master switch to disable Unicode? ;-) Changing the subject... I say -1 on backporting more 3.x features to 2.6/2.7. As previously mentioned, leave the carrot. And best intentions aside, backports can introduce problems of their own.

9 8

PyCon 2010: Poster sessions
by Aahz 04 Nov '09

04 Nov '09

PyCon 2010: Poster sessions =============================================================== Due date: November 30, 2009 PyCon 2010 introduces a new type of presentation, the poster session. Poster sessions consist of two pieces: * A display space where you can put up information about a topic * Live Q&A during a plenary timeslot where people can get more information from you while you stand next to your display For more information and to submit a poster proposal, visit http://us.pycon.org/2010/conference/posters/ -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ [on old computer technologies and programmers] "Fancy tail fins on a brand new '59 Cadillac didn't mean throwing out a whole generation of mechanics who started with model As." --Andrew Dalke

1 0

PEP 382: Namespace Packages
by "Martin v. Löwis" 04 Nov '09

04 Nov '09

I propose the following PEP for inclusion to Python 3.1. Please comment. Regards, Martin Abstract ======== Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. In current Python versions, an algorithm to compute the packages __path__ must be formulated. With the enhancement proposed here, the import machinery itself will construct the list of directories that make up the package. Terminology =========== Within this PEP, the term package refers to Python packages as defined by Python's import statement. The term distribution refers to separately installable sets of Python modules as stored in the Python package index, and installed by distutils or setuptools. The term vendor package refers to groups of files installed by an operating system's packaging mechanism (e.g. Debian or Redhat packages install on Linux systems). The term portion refers to a set of files in a single directory (possibly stored in a zip file) that contribute to a namespace package. Namespace packages today ======================== Python currently provides the pkgutil.extend_path to denote a package as a namespace package. The recommended way of using it is to put:: from pkgutil import extend_path __path__ = extend_path(__path__, __name__) int the package's ``__init__.py``. Every distribution needs to provide the same contents in its ``__init__.py``, so that extend_path is invoked independent of which portion of the package gets imported first. As a consequence, the package's ``__init__.py`` cannot practically define any names as it depends on the order of the package fragments on sys.path which portion is imported first. As a special feature, extend_path reads files named ``*.pkg`` which allow to declare additional portions. setuptools provides a similar function pkg_resources.declare_namespace that is used in the form:: import pkg_resources pkg_resources.declare_namespace(__name__) In the portion's __init__.py, no assignment to __path__ is necessary, as declare_namespace modifies the package __path__ through sys.modules. As a special feature, declare_namespace also supports zip files, and registers the package name internally so that future additions to sys.path by setuptools can properly add additional portions to each package. setuptools allows declaring namespace packages in a distribution's setup.py, so that distribution developers don't need to put the magic __path__ modification into __init__.py themselves. Rationale ========= The current imperative approach to namespace packages has lead to multiple slightly-incompatible mechanisms for providing namespace packages. For example, pkgutil supports ``*.pkg`` files; setuptools doesn't. Likewise, setuptools supports inspecting zip files, and supports adding portions to its _namespace_packages variable, whereas pkgutil doesn't. In addition, the current approach causes problems for system vendors. Vendor packages typically must not provide overlapping files, and an attempt to install a vendor package that has a file already on disk will fail or cause unpredictable behavior. As vendors might chose to package distributions such that they will end up all in a single directory for the namespace package, all portions would contribute conflicting __init__.py files. Specification ============= Rather than using an imperative mechanism for importing packages, a declarative approach is proposed here, as an extension to the existing ``*.pkg`` mechanism. The import statement is extended so that it directly considers ``*.pkg`` files during import; a directory is considered a package if it either contains a file named __init__.py, or a file whose name ends with ".pkg". In addition, the format of the ``*.pkg`` file is extended: a line with the single character ``*`` indicates that the entire sys.path will be searched for portions of the namespace package at the time the namespace packages is imported. Importing a package will immediately compute the package's __path__; the ``*.pkg`` files are not considered anymore after the initial import. If a ``*.pkg`` package contains an asterisk, this asterisk is prepended to the package's __path__ to indicate that the package is a namespace package (and that thus further extensions to sys.path might also want to extend __path__). At most one such asterisk gets prepended to the path. extend_path will be extended to recognize namespace packages according to this PEP, and avoid adding directories twice to __path__. No other change to the importing mechanism is made; searching modules (including __init__.py) will continue to stop at the first module encountered. Discussion ========== With the addition of ``*.pkg`` files to the import mechanism, namespace packages can stop filling out the namespace package's __init__.py. As a consequence, extend_path and declare_namespace become obsolete. It is recommended that distributions put a file <distribution>.pkg into their namespace packages, with a single asterisk. This allows vendor packages to install multiple portions of namespace package into a single directory, with no risk of overlapping files. Namespace packages can start providing non-trivial __init__.py implementations; to do so, it is recommended that a single distribution provides a portion with just the namespace package's __init__.py (and potentially other modules that belong to the namespace package proper). The mechanism is mostly compatible with the existing namespace mechanisms. extend_path will be adjusted to this specification; any other mechanism might cause portions to get added twice to __path__. Copyright ========= This document has been placed in the public domain.

31 110

language summit topic: issue tracker
by Brett Cannon 03 Nov '09

03 Nov '09

Another summit, another potential time to see if people want to change anything about the issue tracker. I would bring up: - Dropping Stage in favor of some keywords (e.g. 'needs unit test', 'needs docs') - Adding a freestyle text box to delineate which, if any, stdlib module is the cause of a bug and tie that into Misc/maintainers.rst; would potentially scale back the Component box -Brett

8 11

thanks to everyone cleaning up the tests
by Brett Cannon 03 Nov '09

03 Nov '09

Just wanted to publicly thank everyone who has been causing all the checkins to fix and stabilize the test suite (I think it's mostly Antoine and Mark, but I could be missing somebody; I'm under a deadline so only have marginal higher brain functionality). -Brett

4 3

A new way to configure logging
by Vinay Sajip 02 Nov '09

02 Nov '09

At present, configuration of Python's logging package can be done in one of two ways: 1. Create a ConfigParser-readable configuration file and use logging.config.fileConfig() to read and implement the configuration therein. 2. Use the logging API to programmatically configure logging using getLogger(), addHandler() etc. The first of these works for simple cases but does not cover all of the logging API (e.g. Filters). The second of these provides maximal control, but besides requiring users to write the configuration code, it fixes the configuration in Python code and does not facilitate changing it easily at runtime. In addition, the ConfigParser format appears to engender dislike (sometimes strong dislike) in some quarters. Though it was chosen because it was the only configuration format supported in the stdlib at that time, many people regard it (or perhaps just the particular schema chosen for logging's configuration) as 'crufty' or 'ugly', in some cases apparently on purely aesthetic grounds. Recent versions of Python of course support an additional battery-included format which can be used for configuration - namely, JSON. Other options, such as YAML, are also possible ways of configuring systems, Google App Engine-style, and PyYAML has matured nicely. There has also been talk on the django-dev mailing list about providing better support for using Python logging in Django. When it happens (as of course I hope it does) this has the consequence that many new users who use Django but are relatively inexperienced in Python (e.g. in PHP shops which are moving to Django) will become exposed to Python logging. As Django is configured using a Python module and use of ConfigParser-style files is not a common approach in that ecosystem, users will find either of the two approaches outlined above a particular pain point when configuring logging for their Django applications and websites, unless something is done to avoid it. All three of the contenders for the title of "commonly found configuration mechanism" - JSON, YAML and Python code - will be expressible, in Python, as Python dicts. So it seems to make sense to add, to logging.config, a new callable bound to "dictConfig" which will take a single dictionary argument and configure logging from that dictionary. An important facet of implementing such a scheme will be the format or schema which the dictionary has to adhere to. I have started working on what such a schema would look like, and if people here think it's a good idea to go ahead with this, I'll provide the details of the schema on a separate post which I'll also cross-post on comp.lang.python so that I can get feedback from there, too. In outline, the scheme I have in mind will look like this, in terms of the new public API: class DictConfigurator: def __init__(self, config): #config is a dict-like object (duck-typed) import copy self.config = copy.deepcopy(config) def configure(self): # actually do the configuration here using self.config dictConfigClass = DictConfigurator def dictConfig(config): dictConfigClass(config).configure() This allows easy replacement of DictConfigurator with a suitable subclass where needed. What's the general feeling here about this proposal? All comments and suggestions will be gratefully received. Regards, Vinay Sajip

6 16

Possible language summit topic: buildbots
by Mark Dickinson 02 Nov '09

02 Nov '09

Would it be worth spending some time discussing the buildbot situation at the PyCon 2010 language summit? In the past, I've found the buildbots to be an incredibly valuable resource; especially when working with aspects of Python or C that tend to vary significantly from platform to platform (for me, this usually means floating-point, and platform math libraries, but there are surely many other things it applies to). But more recently there seem to have been some difficulties keeping a reasonable number of buildbots up and running. A secondary problem is that it can be awkward to debug some of the more obscure test failures on buildbots without having direct access to the machine. From conversations on IRC, I don't think I'm alone in wanting to find ways to make the buildbots more useful. So the question is: how best to invest time and possibly money to improve the buildbot situation (and as a result, I hope, improve the quality of Python)? What could be done to make maintenance of build slaves easier? Or to encourage interested third parties to donate hardware and time? Are there good alternatives to Buildbot that might make a difference? What do other projects do? These are probably the wrong questions; I'm hoping that a discussion would help produce the right questions, and possibly some answers. Mark

15 71

Integer behaviour in Python 2.6.4
by Sturla Molden 02 Nov '09

02 Nov '09

Why does this happen? >>> type(2**31-1) <type 'long'> It seems to have broken NumPy's RNG on Win32.

2 2

Bizarre mtime behaviour
by Antoine Pitrou 01 Nov '09

01 Nov '09

Hello, I wondered if someone had a clue about the following behaviour. While debugging an erratic test_mailbox failure on RDM's buildbot (and other machines), it turned out that the system sometimes set the wrong mtime on a directory: $ date && python -c 'import os; os.link("setup.py", "t/c")' && stat t && date Sun Nov 1 09:49:04 EST 2009 File: `t' Size: 144 Blocks: 0 IO Block: 4096 directory Device: 811h/2065d Inode: 223152 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 1001/ pitrou) Gid: ( 1005/ pitrou) Access: 2009-11-01 09:10:11.000000000 -0500 Modify: 2009-11-01 09:49:03.000000000 -0500 Change: 2009-11-01 09:49:03.000000000 -0500 Sun Nov 1 09:49:04 EST 2009 As you see above, the mtime for directory 't' is set to a full second before the actual modification has happened. Sprinkling traces of time.time() and os.path.getmtime() on Lib/mailbox.py shows this is exactly what trips up test_mailbox. I've got posted a patch to fix it (see issue #6896), but I would like to know if such OS behaviour is normal. Regards Antoine.

4 5