Mailman 3 April 2009 - Python-checkins

r72147 - peps/trunk/pep-0383.txt
by martin.v.loewis 30 Apr '09

30 Apr '09

Author: martin.v.loewis Date: Thu Apr 30 11:24:24 2009 New Revision: 72147 Log: Add more discussion. Modified: peps/trunk/pep-0383.txt Modified: peps/trunk/pep-0383.txt ============================================================================== --- peps/trunk/pep-0383.txt (original) +++ peps/trunk/pep-0383.txt Thu Apr 30 11:24:24 2009 @@ -104,7 +104,10 @@ get converted back to bytes with the python-escape error handler also. Encoding the data with the locale's encoding and the (default) strict error handler will raise an exception, encoding them with UTF-8 -will produce non-sensical data. +will produce non-sensical data. + +Data obtained from other sources may conflict with data produced +by this PEP. Dealing with such conflicts is out of scope of the PEP. For most applications, we assume that they eventually pass data received from a system interface back into the same system @@ -134,6 +137,11 @@ python-escape, there are required byte sequences which cannot be generated from replacement Unicode. +A few alternative approaches have been proposed: + +* create a new string subclass that supports embedded bytes +* use different escape schemes, such as escaping with a NUL + character, or mapping to infrequent characters. References ==========

1 0

r72146 - peps/trunk/pep-0382.txt
by martin.v.loewis 30 Apr '09

30 Apr '09

Author: martin.v.loewis Date: Thu Apr 30 11:05:00 2009 New Revision: 72146 Log: Rename proposed files from .pkg into .pth. Drop claims that this extends the .pkg mechanism. Modified: peps/trunk/pep-0382.txt Modified: peps/trunk/pep-0382.txt ============================================================================== --- peps/trunk/pep-0382.txt (original) +++ peps/trunk/pep-0382.txt Thu Apr 30 11:05:00 2009 @@ -49,8 +49,8 @@ first. As a consequence, the package's ``__init__.py`` cannot practically define any names as it depends on the order of the package fragments on sys.path which portion is imported first. As a special -feature, extend_path reads files named ``*.pkg`` which allow to -declare additional portions. +feature, extend_path reads files named ``<packagename>.pkg`` which +allow to declare additional portions. setuptools provides a similar function pkg_resources.declare_namespace that is used in the form:: @@ -91,29 +91,27 @@ Rather than using an imperative mechanism for importing packages, a declarative approach is proposed here, as an extension to the existing -``*.pkg`` mechanism. +``*.pth`` mechanism available on the top-level python path. -The import statement is extended so that it directly considers ``*.pkg`` -files during import; a directory is considered a package if it either -contains a file named __init__.py, or a file whose name ends with -".pkg". +The import statement is extended so that it directly considers +``*.pth`` files during import; a directory is considered a package if +it either contains a file named __init__.py, or a file whose name ends +with ".pth". Unlike .pth files on the top level, lines starting with +"import" are not supported in per-package .pth files. -In addition, the format of the ``*.pkg`` file is extended: a line with +In addition, the format of the ``*.pth`` file is extended: a line with the single character ``*`` indicates that the entire sys.path will be searched for portions of the namespace package at the time the namespace packages is imported. Importing a package will immediately compute the package's __path__; -the ``*.pkg`` files are not considered anymore after the initial import. -If a ``*.pkg`` package contains an asterisk, this asterisk is prepended -to the package's __path__ to indicate that the package is a namespace -package (and that thus further extensions to sys.path might also -want to extend __path__). At most one such asterisk gets prepended -to the path. In addition, the (possibly dotted) names of all namespace -packages are added to the set sys.namespace_packages. - -extend_path will be extended to recognize namespace packages according -to this PEP, and avoid adding directories twice to __path__. +the ``*.pth`` files are not considered anymore after the initial +import. If a ``*.pth`` package contains an asterisk, this asterisk is +prepended to the package's __path__ to indicate that the package is a +namespace package (and that thus further extensions to sys.path might +also want to extend __path__). At most one such asterisk gets +prepended to the path. In addition, the (possibly dotted) names of all +namespace packages are added to the set sys.namespace_packages. No other change to the importing mechanism is made; searching modules (including __init__.py) will continue to stop at the first module @@ -122,22 +120,22 @@ 1. sys.path is search for a directory foo, or a file foo.<ext>. If a file is found, it is treated as a module, and imported. - 2. if it is a directory, it checks for \*.pkg files. If it finds + 2. if it is a directory, it checks for \*.pth files. If it finds any, a package is created, and its __path__ is extended. 3. The __init__ module is imported; this import will search the __path__ that got computed already. - 4. If neither a \*.pkg file nor an __init__.py was found, the + 4. If neither a \*.pth file nor an __init__.py was found, the directory is skipped, and search for the module/package continues. Discussion ========== -With the addition of ``*.pkg`` files to the import mechanism, namespace +With the addition of ``*.pth`` files to the import mechanism, namespace packages can stop filling out the namespace package's __init__.py. As a consequence, extend_path and declare_namespace become obsolete. -It is recommended that distributions put a file <distribution>.pkg +It is recommended that distributions put a file <distribution>.pth into their namespace packages, with a single asterisk. This allows vendor packages to install multiple portions of namespace package into a single directory, with no risk of overlapping files. @@ -160,9 +158,6 @@ users of this feature are encouraged to comment on this particular question. -It also has been proposed to rename the extension from .pkg to -something else. - Copyright =========

1 0

buildbot failure in ia64 Ubuntu 3.x
by buildbot＠python.org 30 Apr '09

30 Apr '09

The Buildbot has detected a new failure of ia64 Ubuntu 3.x. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%20Ubuntu%203.x/builds/670 Buildbot URL: http://www.python.org/dev/buildbot/all/ Buildslave for this Build: klose-debian-ia64 Build Reason: Build Source Stamp: [branch branches/py3k] HEAD Blamelist: matthias.klose BUILD FAILED: failed test Excerpt from the test logfile: 2 tests failed: test_os test_time ====================================================================== FAIL: test_update2 (test.test_os.EnvironTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/3.x.klose-debian-ia64/build/Lib/test/test_os.py", line 382, in test_update2 self.assertEquals(value, "World") AssertionError: '' != 'World' ====================================================================== FAIL: test_tzset (test.test_time.TimeTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/3.x.klose-debian-ia64/build/Lib/test/test_time.py", line 166, in test_tzset self.failIfEqual(time.gmtime(xmas2002), time.localtime(xmas2002)) AssertionError: time.struct_time(tm_year=2002, tm_mon=12, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=2, tm_yday=359, tm_isdst=0) == time.struct_time(tm_year=2002, tm_mon=12, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=2, tm_yday=359, tm_isdst=0) make: *** [buildbottest] Error 1 sincerely, -The Buildbot

1 0

r72145 - peps/trunk/pep-0383.txt
by martin.v.loewis 30 Apr '09

30 Apr '09

Author: martin.v.loewis Date: Thu Apr 30 10:34:04 2009 New Revision: 72145 Log: Add discussion of error handlers proposed by Glen Linderman. Modified: peps/trunk/pep-0383.txt Modified: peps/trunk/pep-0383.txt ============================================================================== --- peps/trunk/pep-0383.txt (original) +++ peps/trunk/pep-0383.txt Thu Apr 30 10:34:04 2009 @@ -80,7 +80,8 @@ The error handler interface is extended to allow the encode error handler to return byte strings immediately, in addition to returning -Unicode strings which then get encoded again. +Unicode strings which then get encoded again (also see the discussion +below). If the locale's encoding is UTF-8, the file system encoding is set to a new encoding "utf-8b", as the regular UTF-8 codec would not @@ -123,6 +124,17 @@ # fn is now a str object yield fn.encode(fse, "python-escape") +The encode error handler interface presently requires replacement +Unicode to be provide in lieu of the non-encodable Unicode from the +source string. It promptly encodes that replacement Unicode. In some +error handlers, such as the python-escape proposed here, it is simpler +and more efficient for the error handler to provide a pre-encoded +replacement byte string, rather than forcing it to calculating Unicode +from which the encoder would create the desired bytes. In fact, with +python-escape, there are required byte sequences which cannot be +generated from replacement Unicode. + + References ==========

1 0

r72144 - in python/branches/py3k: Misc/NEWS configure configure.in setup.py
by matthias.klose 30 Apr '09

30 Apr '09

Author: matthias.klose Date: Thu Apr 30 10:06:49 2009 New Revision: 72144 Log: - Issue #4587: Add configure option --with-dbmliborder=db1:db2:... to specify the order that backends for the dbm extension are checked. Modified: python/branches/py3k/Misc/NEWS python/branches/py3k/configure python/branches/py3k/configure.in python/branches/py3k/setup.py Modified: python/branches/py3k/Misc/NEWS ============================================================================== --- python/branches/py3k/Misc/NEWS (original) +++ python/branches/py3k/Misc/NEWS Thu Apr 30 10:06:49 2009 @@ -880,6 +880,9 @@ Build ----- +- Issue #4587: Add configure option --with-dbmliborder=db1:db2:... to specify + the order that backends for the dbm extension are checked. + - Link the shared python library with $(MODLIBS). - Issue #5134: Silence compiler warnings when compiling sqlite with VC++. Modified: python/branches/py3k/configure ============================================================================== --- python/branches/py3k/configure (original) +++ python/branches/py3k/configure Thu Apr 30 10:06:49 2009 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 71723 . +# From configure.in Revision: 71731 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.61 for python 3.1. # @@ -1340,6 +1340,10 @@ --with-pydebug build with Py_DEBUG defined --with-libs='lib1 ...' link against additional libs --with-system-ffi build _ctypes module using an installed ffi library + --with-dbmliborder=db1:db2:... + order to check db backends for dbm. Valid value is a + colon separated string with the backend names + `ndbm', `gdbm' and `bdb'. --with-signal-module disable/enable signal module --with-dec-threads use DEC Alpha/OSF1 thread-safe libraries --with(out)-threads[=DIRECTORY] @@ -14087,6 +14091,33 @@ { echo "$as_me:$LINENO: result: $with_system_ffi" >&5 echo "${ECHO_T}$with_system_ffi" >&6; } +# Check for --with-dbmliborder +{ echo "$as_me:$LINENO: checking for --with-dbmliborder" >&5 +echo $ECHO_N "checking for --with-dbmliborder... $ECHO_C" >&6; } + +# Check whether --with-dbmliborder was given. +if test "${with_dbmliborder+set}" = set; then + withval=$with_dbmliborder; +if test x$with_dbmliborder = xyes +then +{ { echo "$as_me:$LINENO: error: proper usage is --with-dbmliborder=db1:db2:..." >&5 +echo "$as_me: error: proper usage is --with-dbmliborder=db1:db2:..." >&2;} + { (exit 1); exit 1; }; } +else + for db in `echo $with_dbmliborder | sed 's/:/ /g'`; do + if test x$db != xndbm && test x$db != xgdbm && test x$db != xbdb + then + { { echo "$as_me:$LINENO: error: proper usage is --with-dbmliborder=db1:db2:..." >&5 +echo "$as_me: error: proper usage is --with-dbmliborder=db1:db2:..." >&2;} + { (exit 1); exit 1; }; } + fi + done +fi +fi + +{ echo "$as_me:$LINENO: result: $with_dbmliborder" >&5 +echo "${ECHO_T}$with_dbmliborder" >&6; } + # Determine if signalmodule should be used. Modified: python/branches/py3k/configure.in ============================================================================== --- python/branches/py3k/configure.in (original) +++ python/branches/py3k/configure.in Thu Apr 30 10:06:49 2009 @@ -1837,6 +1837,24 @@ AC_MSG_RESULT($with_system_ffi) +# Check for --with-dbmliborder +AC_MSG_CHECKING(for --with-dbmliborder) +AC_ARG_WITH(dbmliborder, + AC_HELP_STRING([--with-dbmliborder=db1:db2:...], [order to check db backends for dbm. Valid value is a colon separated string with the backend names `ndbm', `gdbm' and `bdb'.]), +[ +if test x$with_dbmliborder = xyes +then +AC_MSG_ERROR([proper usage is --with-dbmliborder=db1:db2:...]) +else + for db in `echo $with_dbmliborder | sed 's/:/ /g'`; do + if test x$db != xndbm && test x$db != xgdbm && test x$db != xbdb + then + AC_MSG_ERROR([proper usage is --with-dbmliborder=db1:db2:...]) + fi + done +fi]) +AC_MSG_RESULT($with_dbmliborder) + # Determine if signalmodule should be used. AC_SUBST(USE_SIGNAL_MODULE) AC_SUBST(SIGNAL_OBJS) Modified: python/branches/py3k/setup.py ============================================================================== --- python/branches/py3k/setup.py (original) +++ python/branches/py3k/setup.py Thu Apr 30 10:06:49 2009 @@ -913,37 +913,69 @@ # The standard Unix dbm module: if platform not in ['cygwin']: - if find_file("ndbm.h", inc_dirs, []) is not None: - # Some systems have -lndbm, others don't - if self.compiler.find_library_file(lib_dirs, 'ndbm'): - ndbm_libs = ['ndbm'] - else: - ndbm_libs = [] - exts.append( Extension('_dbm', ['_dbmmodule.c'], - define_macros=[('HAVE_NDBM_H',None)], - libraries = ndbm_libs ) ) - elif self.compiler.find_library_file(lib_dirs, 'gdbm'): - gdbm_libs = ['gdbm'] - if self.compiler.find_library_file(lib_dirs, 'gdbm_compat'): - gdbm_libs.append('gdbm_compat') - if find_file("gdbm/ndbm.h", inc_dirs, []) is not None: - exts.append( Extension( - '_dbm', ['_dbmmodule.c'], - define_macros=[('HAVE_GDBM_NDBM_H',None)], - libraries = gdbm_libs ) ) - elif find_file("gdbm-ndbm.h", inc_dirs, []) is not None: - exts.append( Extension( - '_dbm', ['_dbmmodule.c'], - define_macros=[('HAVE_GDBM_DASH_NDBM_H',None)], - libraries = gdbm_libs ) ) - elif db_incs is not None: - exts.append( Extension('_dbm', ['_dbmmodule.c'], - library_dirs=dblib_dir, - runtime_library_dirs=dblib_dir, - include_dirs=db_incs, - define_macros=[('HAVE_BERKDB_H',None), - ('DB_DBM_HSEARCH',None)], - libraries=dblibs)) + config_args = [arg.strip("'") + for arg in sysconfig.get_config_var("CONFIG_ARGS").split()] + dbm_args = [arg.split('=')[-1] for arg in config_args + if arg.startswith('--with-dbmliborder=')] + if dbm_args: + dbm_order = dbm_args[-1].split(":") + else: + dbm_order = "ndbm:gdbm:bdb".split(":") + dbmext = None + for cand in dbm_order: + if cand == "ndbm": + if find_file("ndbm.h", inc_dirs, []) is not None: + # Some systems have -lndbm, others don't + if self.compiler.find_library_file(lib_dirs, 'ndbm'): + ndbm_libs = ['ndbm'] + else: + ndbm_libs = [] + print("building dbm using ndbm") + dbmext = Extension('_dbm', ['_dbmmodule.c'], + define_macros=[ + ('HAVE_NDBM_H',None), + ], + libraries=ndbm_libs) + break + + elif cand == "gdbm": + if self.compiler.find_library_file(lib_dirs, 'gdbm'): + gdbm_libs = ['gdbm'] + if self.compiler.find_library_file(lib_dirs, 'gdbm_compat'): + gdbm_libs.append('gdbm_compat') + if find_file("gdbm/ndbm.h", inc_dirs, []) is not None: + print("building dbm using gdbm") + dbmext = Extension( + '_dbm', ['_dbmmodule.c'], + define_macros=[ + ('HAVE_GDBM_NDBM_H', None), + ], + libraries = gdbm_libs) + break + if find_file("gdbm-ndbm.h", inc_dirs, []) is not None: + print("building dbm using gdbm") + dbmext = Extension( + '_dbm', ['_dbmmodule.c'], + define_macros=[ + ('HAVE_GDBM_DASH_NDBM_H', None), + ], + libraries = gdbm_libs) + break + elif cand == "bdb": + if db_incs is not None: + print("building dbm using bdb") + dbmext = Extension('_dbm', ['_dbmmodule.c'], + library_dirs=dblib_dir, + runtime_library_dirs=dblib_dir, + include_dirs=db_incs, + define_macros=[ + ('HAVE_BERKDB_H', None), + ('DB_DBM_HSEARCH', None), + ], + libraries=dblibs) + break + if dbmext is not None: + exts.append(dbmext) else: missing.append('_dbm')

1 0

r72143 - peps/trunk/pep-0383.txt
by martin.v.loewis 30 Apr '09

30 Apr '09

Author: martin.v.loewis Date: Thu Apr 30 09:02:13 2009 New Revision: 72143 Log: Add references. Modified: peps/trunk/pep-0383.txt Modified: peps/trunk/pep-0383.txt ============================================================================== --- peps/trunk/pep-0383.txt (original) +++ peps/trunk/pep-0383.txt Thu Apr 30 09:02:13 2009 @@ -64,18 +64,19 @@ On Windows, Python uses the wide character APIs to access character-oriented APIs, allowing direct conversion of the -environmental data to Python str objects. +environmental data to Python str objects ([1]). On POSIX systems, Python currently applies the locale's encoding to convert the byte data to Unicode, failing for characters that cannot be decoded. With this PEP, non-decodable bytes will be represented as lone half surrogate codes U+DCxx. -To convert non-decodable bytes, a new error handler "python-escape" is -introduced, which produces these half surrogates. On encoding, the -error handler converts the half surrogate back to the corresponding -byte. This error handler will be used in any API that receives or -produces file names, command line arguments, or environment variables. +To convert non-decodable bytes, a new error handler ([2]) +"python-escape" is introduced, which produces these half +surrogates. On encoding, the error handler converts the half surrogate +back to the corresponding byte. This error handler will be used in any +API that receives or produces file names, command line arguments, or +environment variables. The error handler interface is extended to allow the encode error handler to return byte strings immediately, in addition to returning @@ -122,6 +123,17 @@ # fn is now a str object yield fn.encode(fse, "python-escape") +References +========== + + [1] PEP 277 + "Unicode file name support for Windows NT" + http://www.python.org/dev/peps/pep-0277/ + + [2] PEP 293 + "Codec Error Handling Callbacks" + http://www.python.org/dev/peps/pep-0293/ + Copyright =========

1 0

r72142 - peps/trunk/pep-0383.txt
by martin.v.loewis 30 Apr '09

30 Apr '09

Author: martin.v.loewis Date: Thu Apr 30 07:04:13 2009 New Revision: 72142 Log: Fix typo. Modified: peps/trunk/pep-0383.txt Modified: peps/trunk/pep-0383.txt ============================================================================== --- peps/trunk/pep-0383.txt (original) +++ peps/trunk/pep-0383.txt Thu Apr 30 07:04:13 2009 @@ -74,8 +74,8 @@ To convert non-decodable bytes, a new error handler "python-escape" is introduced, which produces these half surrogates. On encoding, the error handler converts the half surrogate back to the corresponding -byte. This error handler used used in any API that receives or produces -file names, command line arguments, or environment variables. +byte. This error handler will be used in any API that receives or +produces file names, command line arguments, or environment variables. The error handler interface is extended to allow the encode error handler to return byte strings immediately, in addition to returning

1 0

buildbot failure in sparc solaris10 gcc trunk
by buildbot＠python.org 30 Apr '09

30 Apr '09

The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%20trunk/buil… Buildbot URL: http://www.python.org/dev/buildbot/all/ Buildslave for this Build: loewis-sun Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: benjamin.peterson,eric.smith,georg.brandl,mark.dickinson BUILD FAILED: failed test Excerpt from the test logfile: 1 test failed: test_time ====================================================================== FAIL: test_tzset (test.test_time.TimeTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_time.py", line 152, in test_tzset time.gmtime(xmas2002), time.localtime(xmas2002) AssertionError: time.struct_time(tm_year=2002, tm_mon=12, tm_mday=25, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=2, tm_yday=359, tm_isdst=0) != time.struct_time(tm_year=2002, tm_mon=12, tm_mday=25, tm_hour=1, tm_min=0, tm_sec=0, tm_wday=2, tm_yday=359, tm_isdst=0) sincerely, -The Buildbot

1 0

buildbot failure in ppc Debian unstable trunk
by buildbot＠python.org 30 Apr '09

30 Apr '09

The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%20Debian%20unstable%20trunk/buil… Buildbot URL: http://www.python.org/dev/buildbot/all/ Buildslave for this Build: klose-debian-ppc Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: benjamin.peterson,georg.brandl BUILD FAILED: failed test Excerpt from the test logfile: 2 tests failed: test_bsddb3 test_os make: *** [buildbottest] Error 1 sincerely, -The Buildbot

1 0

buildbot failure in sparc solaris10 gcc 3.0
by buildbot＠python.org 30 Apr '09

30 Apr '09

The Buildbot has detected a new failure of sparc solaris10 gcc 3.0. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%20solaris10%20gcc%203.0/builds… Buildbot URL: http://www.python.org/dev/buildbot/all/ Buildslave for this Build: loewis-sun Build Reason: Build Source Stamp: [branch branches/release30-maint] HEAD Blamelist: r.david.murray BUILD FAILED: failed test Excerpt from the test logfile: 2 tests failed: test_posix test_subprocess ====================================================================== ERROR: test_getcwd_long_pathnames (test.test_posix.PosixTester) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/3.0.loewis-sun/build/Lib/test/test_posix.py", line 252, in test_getcwd_long_pathnames support.rmtree(base_path) File "/home2/buildbot/slave/3.0.loewis-sun/build/Lib/test/support.py", line 98, in rmtree shutil.rmtree(path) File "/home2/buildbot/slave/3.0.loewis-sun/build/Lib/shutil.py", line 225, in rmtree onerror(os.rmdir, path, sys.exc_info()) File "/home2/buildbot/slave/3.0.loewis-sun/build/Lib/shutil.py", line 223, in rmtree os.rmdir(path) OSError: [Errno 22] Invalid argument: '/home2/buildbot/slave/3.0.loewis-sun/build/@test.getcwd/@test.getcwd' ====================================================================== FAIL: test_executable (test.test_subprocess.ProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/3.0.loewis-sun/build/Lib/test/test_subprocess.py", line 115, in test_executable self.assertEqual(p.returncode, 47) AssertionError: -6 != 47 sincerely, -The Buildbot

1 0