Mailman 3 March 2012 - Python-Dev

Summary of Python tracker Issues
by Python tracker March 2, 2012

March 2, 2012

ACTIVITY SUMMARY (2012-02-24 - 2012-03-02) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3299 (+22) closed 22660 (+49) total 25959 (+71) Open issues with patches: 1409 Issues opened (57) ================== #12151: test_logging fails sometimes http://bugs.python.org/issue12151 reopened by nadeem.vawda #14080: Sporadic test_imp failure http://bugs.python.org/issue14080 reopened by skrah #14110: FreeBSD: test_os fails if user is in the wheel group http://bugs.python.org/issue14110 opened by skrah #14111: IDLE Debugger should handle interrupts http://bugs.python.org/issue14111 opened by ltaylor934 #14112: tutorial intro talks of "shallow copy" concept without explana http://bugs.python.org/issue14112 opened by tshepang #14114: 2.7.3rc1 chm gives JS error http://bugs.python.org/issue14114 opened by loewis #14115: 2.7.3rc hangs on test_asynchat on 32-bit Windows http://bugs.python.org/issue14115 opened by loewis #14116: Lock.__enter__() method returns True instead of self http://bugs.python.org/issue14116 opened by sbt #14117: Turtledemo: exception and minor glitches. http://bugs.python.org/issue14117 opened by terry.reedy #14119: Ability to adjust queue size in Executors http://bugs.python.org/issue14119 opened by Nam.Nguyen #14120: ARM Ubuntu 3.x buildbot failing test_dbm http://bugs.python.org/issue14120 opened by nadeem.vawda #14121: add a convenience C-API function for unpacking iterables http://bugs.python.org/issue14121 opened by scoder #14122: operator: div() instead of truediv() in documention since 3.1. http://bugs.python.org/issue14122 opened by felixantoinefortin #14123: Indicate that there are no current plans to deprecate printf-s http://bugs.python.org/issue14123 opened by telephonebook #14124: _pickle.c comment/documentation improvement http://bugs.python.org/issue14124 opened by valhallasw #14126: Speed up list comprehensions by preallocating the list where p http://bugs.python.org/issue14126 opened by alex #14127: os.stat and os.utime: allow preserving exact metadata http://bugs.python.org/issue14127 opened by larry #14128: _elementtree should expose types and factory functions consist http://bugs.python.org/issue14128 opened by eli.bendersky #14130: memoryview: add multi-dimensional indexing and slicing http://bugs.python.org/issue14130 opened by skrah #14131: test_threading failure on WIndows 7 3.x buildbot http://bugs.python.org/issue14131 opened by ncoghlan #14132: Redirect is not working correctly in urllib2 http://bugs.python.org/issue14132 opened by janik #14133: improved PEP 409 implementation http://bugs.python.org/issue14133 opened by benjamin.peterson #14134: xmlrpc.client.ServerProxy needs timeout parameter http://bugs.python.org/issue14134 opened by polymorphm #14135: check for locale changes in test.regrtest http://bugs.python.org/issue14135 opened by brett.cannon #14136: Simplify PEP 409 command line test and move it to test_cmd_lin http://bugs.python.org/issue14136 opened by ncoghlan #14139: test_ftplib: segfault http://bugs.python.org/issue14139 opened by skrah #14140: packaging tests: add helpers to create and inspect a tree of f http://bugs.python.org/issue14140 opened by eric.araujo #14141: 2.7.2 64-bit Windows library has __impt_Py* for several symbol http://bugs.python.org/issue14141 opened by Steve.McConnel #14142: getlocale(LC_ALL) behavior http://bugs.python.org/issue14142 opened by skrah #14143: test_ntpath failure on Windows http://bugs.python.org/issue14143 opened by nadeem.vawda #14144: urllib2 HTTPRedirectHandler not forwarding POST data in redire http://bugs.python.org/issue14144 opened by crustymonkey #14146: IDLE: source line in editor doesn't highlight when debugging http://bugs.python.org/issue14146 opened by Rich.Rauenzahn #14148: Option to kill "stuck" workers in a multiprocessing pool http://bugs.python.org/issue14148 opened by pmoore #14149: argparse: Document how to use argument names that are not Pyth http://bugs.python.org/issue14149 opened by Joseph.Birr-Pixton #14150: AIX, crash loading shared module into another process than pyt http://bugs.python.org/issue14150 opened by Jan.Stürtz #14151: multiprocessing.connection.Listener fails with invalid address http://bugs.python.org/issue14151 opened by Popa.Claudiu #14154: reimplement the bigmem test memory watchdog as a subprocess http://bugs.python.org/issue14154 opened by neologix #14156: argparse.FileType for '-' doesn't work for a mode of 'rb' http://bugs.python.org/issue14156 opened by anacrolix #14157: time.strptime without a year fails on Feb 29 http://bugs.python.org/issue14157 opened by Martin.Morrison #14158: test_mailbox fails if file or dir named by support.TESTFN exis http://bugs.python.org/issue14158 opened by vinay.sajip #14160: TarFile.extractfile fails to extract targets of top-level rela http://bugs.python.org/issue14160 opened by Matthew.Miller #14161: python2 file __repr__ does not escape filename http://bugs.python.org/issue14161 opened by Ronny.Pfannschmidt #14162: PEP 416: Add a builtin frozendict type http://bugs.python.org/issue14162 opened by haypo #14163: tkinter: problems with hello doc example http://bugs.python.org/issue14163 opened by terry.reedy #14166: private dispatch table for picklers http://bugs.python.org/issue14166 opened by sbt #14167: document return statement in finally blocks http://bugs.python.org/issue14167 opened by Yury.Selivanov #14168: Bug in minidom 3.3 after optimization patch http://bugs.python.org/issue14168 opened by vinay.sajip #14169: compiler.compile fails on "if" statement in attached file http://bugs.python.org/issue14169 opened by menegazzobr #14170: print unicode string error in win8 cmd console http://bugs.python.org/issue14170 opened by nkxyz #14171: warnings from valgrind about openssl as used by CPython http://bugs.python.org/issue14171 opened by zooko #14172: ref-counting leak in buffer usage in Python/marshal.c http://bugs.python.org/issue14172 opened by scoder #14173: PyOS_FiniInterupts leaves signal.getsignal segfaulty http://bugs.python.org/issue14173 opened by ferringb #14174: argparse.REMAINDER fails to parse remainder correctly http://bugs.python.org/issue14174 opened by rr2do2 #14176: Fix unicode literals (for PEP 414) http://bugs.python.org/issue14176 opened by Jean-Michel.Fauth #14177: marshal.loads accepts unicode strings http://bugs.python.org/issue14177 opened by pitrou #14178: Failing tests for ElementTree http://bugs.python.org/issue14178 opened by scoder #1346572: Remove inconsistent behavior between import and zipimport http://bugs.python.org/issue1346572 reopened by eric.araujo Most recent 15 issues with no replies (15) ========================================== #14178: Failing tests for ElementTree http://bugs.python.org/issue14178 #14177: marshal.loads accepts unicode strings http://bugs.python.org/issue14177 #14174: argparse.REMAINDER fails to parse remainder correctly http://bugs.python.org/issue14174 #14171: warnings from valgrind about openssl as used by CPython http://bugs.python.org/issue14171 #14169: compiler.compile fails on "if" statement in attached file http://bugs.python.org/issue14169 #14166: private dispatch table for picklers http://bugs.python.org/issue14166 #14160: TarFile.extractfile fails to extract targets of top-level rela http://bugs.python.org/issue14160 #14151: multiprocessing.connection.Listener fails with invalid address http://bugs.python.org/issue14151 #14143: test_ntpath failure on Windows http://bugs.python.org/issue14143 #14142: getlocale(LC_ALL) behavior http://bugs.python.org/issue14142 #14141: 2.7.2 64-bit Windows library has __impt_Py* for several symbol http://bugs.python.org/issue14141 #14140: packaging tests: add helpers to create and inspect a tree of f http://bugs.python.org/issue14140 #14135: check for locale changes in test.regrtest http://bugs.python.org/issue14135 #14130: memoryview: add multi-dimensional indexing and slicing http://bugs.python.org/issue14130 #14126: Speed up list comprehensions by preallocating the list where p http://bugs.python.org/issue14126 Most recent 15 issues waiting for review (15) ============================================= #14172: ref-counting leak in buffer usage in Python/marshal.c http://bugs.python.org/issue14172 #14167: document return statement in finally blocks http://bugs.python.org/issue14167 #14166: private dispatch table for picklers http://bugs.python.org/issue14166 #14163: tkinter: problems with hello doc example http://bugs.python.org/issue14163 #14162: PEP 416: Add a builtin frozendict type http://bugs.python.org/issue14162 #14161: python2 file __repr__ does not escape filename http://bugs.python.org/issue14161 #14158: test_mailbox fails if file or dir named by support.TESTFN exis http://bugs.python.org/issue14158 #14154: reimplement the bigmem test memory watchdog as a subprocess http://bugs.python.org/issue14154 #14151: multiprocessing.connection.Listener fails with invalid address http://bugs.python.org/issue14151 #14150: AIX, crash loading shared module into another process than pyt http://bugs.python.org/issue14150 #14144: urllib2 HTTPRedirectHandler not forwarding POST data in redire http://bugs.python.org/issue14144 #14136: Simplify PEP 409 command line test and move it to test_cmd_lin http://bugs.python.org/issue14136 #14134: xmlrpc.client.ServerProxy needs timeout parameter http://bugs.python.org/issue14134 #14133: improved PEP 409 implementation http://bugs.python.org/issue14133 #14132: Redirect is not working correctly in urllib2 http://bugs.python.org/issue14132 Top 10 most discussed issues (10) ================================= #8706: accept keyword arguments on most base type methods and builtin http://bugs.python.org/issue8706 18 msgs #14080: Sporadic test_imp failure http://bugs.python.org/issue14080 17 msgs #11379: Remove "lightweight" from minidom description http://bugs.python.org/issue11379 16 msgs #14133: improved PEP 409 implementation http://bugs.python.org/issue14133 15 msgs #13405: Add DTrace probes http://bugs.python.org/issue13405 13 msgs #14127: os.stat and os.utime: allow preserving exact metadata http://bugs.python.org/issue14127 13 msgs #14112: tutorial intro talks of "shallow copy" concept without explana http://bugs.python.org/issue14112 11 msgs #1346572: Remove inconsistent behavior between import and zipimport http://bugs.python.org/issue1346572 9 msgs #2377: Replace __import__ w/ importlib.__import__ http://bugs.python.org/issue2377 8 msgs #14097: Improve the "introduction" page of the tutorial http://bugs.python.org/issue14097 8 msgs Issues closed (48) ================== #2394: [Py3k] Finish the memoryview object implementation http://bugs.python.org/issue2394 closed by skrah #2945: bdist_rpm does not list dist files (should effect upload) http://bugs.python.org/issue2945 closed by eric.araujo #6210: Exception Chaining missing method for suppressing context http://bugs.python.org/issue6210 closed by ncoghlan #9845: Allow changing the method in urllib.request.Request http://bugs.python.org/issue9845 closed by eric.araujo #10181: Problems with Py_buffer management in memoryobject.c (and else http://bugs.python.org/issue10181 closed by skrah #10713: re module doesn't describe string boundaries for \b http://bugs.python.org/issue10713 closed by ezio.melotti #11457: os.stat(): add new fields to get timestamps as Decimal objects http://bugs.python.org/issue11457 closed by larry #12903: test_io.test_interrupte[r]d* blocks on OpenBSD http://bugs.python.org/issue12903 closed by neologix #12904: Change os.utime &c functions to use nanosecond precision where http://bugs.python.org/issue12904 closed by larry #12905: multiple errors in test_socket on OpenBSD http://bugs.python.org/issue12905 closed by neologix #13053: Add Capsule migration documentation to "cporting" http://bugs.python.org/issue13053 closed by larry #13086: Update howto/cporting.rst so it talks about Python 3 instead o http://bugs.python.org/issue13086 closed by larry #13125: test_all_project_files() expected failure http://bugs.python.org/issue13125 closed by pitrou #13167: Add get_metadata to packaging http://bugs.python.org/issue13167 closed by eric.araujo #13447: Add tests for some scripts in Tools/scripts http://bugs.python.org/issue13447 closed by eric.araujo #13491: Fixes for sqlite3 doc http://bugs.python.org/issue13491 closed by petri.lehtinen #13521: Make dict.setdefault() atomic http://bugs.python.org/issue13521 closed by pitrou #13706: non-ascii fill characters no longer work in formatting http://bugs.python.org/issue13706 closed by haypo #13716: distutils doc contains lots of XXX http://bugs.python.org/issue13716 closed by eric.araujo #13770: python3 & json: add ensure_ascii documentation http://bugs.python.org/issue13770 closed by eric.araujo #13873: SIGBUS in test_big_buffer() of test_zlib on Debian bigmem buil http://bugs.python.org/issue13873 closed by nadeem.vawda #13973: urllib.parse is imported twice in xmlrpc.client http://bugs.python.org/issue13973 closed by eric.araujo #13998: Lookbehind assertions go behind the start position for the mat http://bugs.python.org/issue13998 closed by ezio.melotti #13999: Queue references in multiprocessing doc points to Queue module http://bugs.python.org/issue13999 closed by sandro.tosi #14049: execfile() fails on files that use global variables inside fun http://bugs.python.org/issue14049 closed by terry.reedy #14081: Allow "maxsplit" argument to str.split() to be passed as a key http://bugs.python.org/issue14081 closed by ezio.melotti #14089: Patch to increase fractions lib test coverage http://bugs.python.org/issue14089 closed by ezio.melotti #14092: __name__ inconsistently applied in class definition http://bugs.python.org/issue14092 closed by terry.reedy #14095: type_new() removes __qualname__ from the input dictionary http://bugs.python.org/issue14095 closed by python-dev #14103: argparse: add ability to create a bash completion script http://bugs.python.org/issue14103 closed by eric.araujo #14107: Debian bigmem buildbot hanging in test_bigmem http://bugs.python.org/issue14107 closed by neologix #14108: test_shutil: failures in symlink tests http://bugs.python.org/issue14108 closed by pitrou #14109: test_lib2to3: output that looks like a failure on Windows 7 http://bugs.python.org/issue14109 closed by pitrou #14113: Failure in test_strptime on Windows http://bugs.python.org/issue14113 closed by nadeem.vawda #14118: _pickle.c structure cleanup http://bugs.python.org/issue14118 closed by loewis #14125: Windows: failures in refleak mode http://bugs.python.org/issue14125 closed by skrah #14129: Corrections for the "extending" doc http://bugs.python.org/issue14129 closed by python-dev #14137: GTK3 Segmentation fault from Warning: g_object_notify: asserti http://bugs.python.org/issue14137 closed by neologix #14138: Ctrl-C does not terminate GTK3 Gtk.main() loop when program ru http://bugs.python.org/issue14138 closed by neologix #14145: string.rfind() returns AttributeError: 'list' object has no at http://bugs.python.org/issue14145 closed by eric.smith #14147: print r"\" cause SyntaxError http://bugs.python.org/issue14147 closed by ezio.melotti #14152: setup.py: Python header file dependencies http://bugs.python.org/issue14152 closed by eric.araujo #14153: Expose os.device_encoding() at the C level http://bugs.python.org/issue14153 closed by brett.cannon #14155: Deja vu in re's documentation http://bugs.python.org/issue14155 closed by ezio.melotti #14159: __len__ method of weakset http://bugs.python.org/issue14159 closed by pitrou #14164: Hyphenation suggestions - floating-point/floating point http://bugs.python.org/issue14164 closed by brian.curtin #14165: The new shlex.quote() function should be marked "New in versio http://bugs.python.org/issue14165 closed by python-dev #14175: broken links on /download/ page http://bugs.python.org/issue14175 closed by georg.brandl

1 0

Compiling Python on Linux with Intel's icc
by Alex Leach March 2, 2012

March 2, 2012

Dear Python Devs, I've been attempting to compile a fully functional version of Python 2.7 using Intel's C compiler, having built supposedly optimal versions of numpy and scipy, using Intel Composer XE and Intel's Math Kernel Library. I can build a working Python binary, but I'd really appreciate if someone could check my compile options, and perhaps suggest ways I could further optimise the build. *** COMPILE FAILURE - ffi64.c *** I've managed to compile everything in the python distribution except for Modules/_ctypes/libffi/src/x86/ffi64.c. So to get the compilation to actually work, I've had to use the config option '--with-system-ffi'. If someone could suggest a patch for ffi64.c, I'd happily test it, as I've been unable to fix the code myself! The problem is with register_args, which uses GCC's __int128_t, but this doesn't exist when using icc. The include guard to use could be:- #ifdef __INTEL_COMPILER ... #else ... #endif I've tried using this guard around the register_args struct, at the top of ffi64.c, and where I see register_args used, around lines 592-616, according to the suggestion at http://software.intel.com/en- us/forums/showthread.php?t=56652, but have been unable to get a working solution... A patch would be appreciated! *** Tests *** After compilation, there's a few tests that are consistently failing, mainly involved with floating point precision: test_cmath, test_math and test_float. Also, I wrote a very short script to test the time of for loop execution and integer multiplication. This script (below) has nearly always completed faster using the default Ubuntu Python rather than my own build. Obviously, I was hoping to get a faster python, but the size of the final binary is almost twice the size of the default Ubuntu version (5.2MB cf. 2.7MB), which I thought might cause a startup overhead that leads to slower execution times when running such a basic script. *** TEST SCRIPT *** $ cat ~/bin/timetest.py RANGE = 10000 print "running {0}^2 = {1} for loop iterations".format( RANGE,RANGE**2 ) for i in xrange(RANGE): for j in xrange(RANGE): i * j *** TIMES *** ## ICC-compiled python ## $ time ./python ~/bin/timetest.py running 10000^2 = 100000000 for loop iterations real 0m2.767s user 0m2.720s sys 0m0.008s ## System python ## $ time python ~/bin/timetest.py running 10000^2 = 100000000 for loop iterations real 0m2.781s user 0m2.776s sys 0m0.000s Oh... My python appears to run faster than gcc's now - checked this a few times now, mine's staying faster... :) I've compiled and re-compiled python dozens of times now, but it's still failing some tests... *** Build Environment *** Ubuntu 10.10 server kernel (`uname -r`=3.0.0-16-server) with KDE 4.7.4 $ tail ~/.bashrc #### Custom Commands export PATH=$PATH:/usr/local/cuda/bin:$HOME/bin export PYTHONPATH=$HOME/bin:/usr/lib/pymodules/python2.7 export PYTHONSTARTUP=$HOME/.pystartup export LD_LIBRARY_PATH=/lib64:/usr/lib64:/usr/local/lib:/usr/local/cuda/lib64:/usr/local/cuda/lib # Load Intel compiler and library variables. source /usr/intel/bin/compilervars.sh intel64 source /usr/intel/impi/4.0.3/bin/mpivars.sh intel64 source /usr/intel/tbb/bin/tbbvars.sh intel64 $ env | grep 'PATH\|FLAGS' MANPATH=/usr/intel/impi/4.0.3.008/man:/usr/intel/composer_xe_2011_sp1.9.293/man/en_US:/usr/intel/composer_xe_2011_sp1.9.293/man/en_US:/usr/intel/impi/4.0.3.008/man:/usr/intel/composer_xe_2011_sp1.9.293/man/en_US:/usr/intel/composer_xe_2011_sp1.9.293/man/en_US:/usr/intel/impi/4.0.3.008/man:/usr/intel/composer_xe_2011_sp1.9.293/man/en_US:/usr/intel/composer_xe_2011_sp1.9.293/man/en_US:/usr/local/man:/usr/local/share/man:/usr/share/man:/usr/intel/man::: LIBRARY_PATH=/usr/intel/composer_xe_2011_sp1.9.293/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:/usr/intel/composer_xe_2011_sp1.9.293/compiler/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/ipp/../compiler/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/ipp/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/compiler/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/mkl/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21 FPATH=/usr/intel/composer_xe_2011_sp1.9.293/mkl/include:/usr/intel/composer_xe_2011_sp1.9.293/mkl/include LD_LIBRARY_PATH=/usr/intel/composer_xe_2011_sp1.9.293/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:/usr/intel/impi/4.0.3.008/ia32/lib:/usr/intel/composer_xe_2011_sp1.9.293/compiler/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/ipp/../compiler/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/ipp/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/compiler/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/mkl/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:/biol/arb/lib:/lib64:/usr/lib64:/usr/local/lib:/usr/local/cuda/lib64:/usr/local/cuda/lib:/usr/intel/composer_xe_2011_sp1.9.293/debugger/lib/intel64:/usr/intel/composer_xe_2011_sp1.9.293/mpirt/lib/intel64 CPATH=/usr/intel/composer_xe_2011_sp1.9.293/tbb/include:/usr/intel/composer_xe_2011_sp1.9.293/mkl/include:/usr/intel/composer_xe_2011_sp1.9.293/tbb/include NLSPATH=/usr/intel/composer_xe_2011_sp1.9.293/compiler/lib/intel64/locale/%l_%t/%N:/usr/intel/composer_xe_2011_sp1.9.293/ipp/lib/intel64/locale/%l_%t/%N:/usr/intel/composer_xe_2011_sp1.9.293/mkl/lib/intel64/locale/%l_%t/%N:/usr/intel/composer_xe_2011_sp1.9.293/debugger/intel64/locale/%l_%t/%N PATH=/usr/intel/impi/4.0.3.008/ia32/bin:/usr/intel/composer_xe_2011_sp1.9.293/bin/intel64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/intel/bin:/usr/local/cuda/bin:/usr/local/cuda/bin:/usr/intel/composer_xe_2011_sp1.9.293/mpirt/bin/intel64 PYTHONPATH=/usr/lib/pymodules/python2.7/ WINDOWPATH=7 QT_PLUGIN_PATH=$HOME/.kde/lib/kde4/plugins/:/usr/lib/kde4/plugins/ *** Download, configure and Build instructions *** $ hg clone -r 2.7 http://hg.python.org/cpython Since... $ hg update -r 2.7 *** Generate Profile-Guided Optimisation stuff with first build *** $ make distclean && mkdir PGO $ CC=icc AR=xiar LD=xild CXX=icpc \ CPPFLAGS+="-I/usr/include \ -I/usr/include/x86_86-linux-gnu \ -I/usr/src/linux-headers-3.0.0-16-server/include/" \ CFLAGS+="-O3 \ -fomit-frame-pointer \ -shared-intel \ -fpic \ -prof-gen \ -prof-dir $PWD/PGO \ -fp-model precise \ -fp-model source \ -xHost \ -ftz" ./configure --with-system-ffi --with-libc="-lirc" --with-libm="-limf" $ make -j9 *** Use the PGO-generated information in new build *** $ make clean $ CC=icc AR=xiar LD=xild CXX=icpc \ CPPFLAGS+="-I/usr/include \ -I/usr/include/x86_86-linux-gnu \ -I/usr/src/linux-headers-3.0.0-16-server/include/" \ CFLAGS+="-O3 \ -fomit-frame-pointer \ -shared-intel \ -fpic \ -prof-use \ -prof-dir $PWD/PGO \ -fp-model precise \ -fp-model source \ -xHost \ -ftz \ -fomit-frame-pointer" \ ./configure --with-system-ffi --with-libc="-lirc" --with-libm="-limf" $ make -j9 ... $ make test building dbm using gdbm Python build finished, but the necessary bits to build these modules were not found: _bsddb bsddb185 dl imageop sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name. find ./Lib -name '*.py[co]' -print | xargs rm -f ./python -Wd -3 -E -tt ./Lib/test/regrtest.py -l /usr/local/src/pysrc/cpython/Lib/unittest/util.py:2: ImportWarning: Not importing directory '/usr/local/src/pysrc/cpython/Lib/collections': missing __init__.py from collections import namedtuple, OrderedDict == CPython 2.7.3rc1 (2.7:5c52e7c6d868+, Feb 29 2012, 22:10:22) [GCC Intel(R) C++ gcc 4.6 mode] == Linux-3.0.0-16-server-x86_64-with-debian-wheezy-sid little-endian == /usr/local/src/pysrc/cpython/build/test_python_16278 Testing with flags: sys.flags(debug=0, py3k_warning=1, division_warning=1, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, tabcheck=2, verbose=0, unicode=0, bytes_warning=0, hash_randomization=0) ......... test_cmath test test_cmath failed -- Traceback (most recent call last): File "/usr/local/src/pysrc/cpython/Lib/test/test_cmath.py", line 352, in test_specific_values msg=error_message) File "/usr/local/src/pysrc/cpython/Lib/test/test_cmath.py", line 94, in rAssertAlmostEqual 'got {!r}'.format(a, b)) AssertionError: acos0000: acos(complex(0.0, 0.0)) Expected: complex(1.5707963267948966, -0.0) Received: complex(1.5707963267948966, 0.0) Received value insufficiently close to expected value. ... test_curses skipped -- Use of the `curses' resource not enabled ... test_float test test_float failed -- Traceback (most recent call last): File "/usr/local/src/pysrc/cpython/Lib/test/test_float.py", line 1273, in test_from_hex self.identical(fromHex('0x0.ffffffffffffd6p-1022'), MIN-3*TINY) File "/usr/local/src/pysrc/cpython/Lib/test/test_float.py", line 983, in identical self.fail('%r not identical to %r' % (x, y)) AssertionError: 0.0 not identical to 2.2250738585072014e-308 ..... test test_strtod failed -- multiple errors occurred; run in verbose mode for details ...... 347 tests OK. 5 tests failed: test_cmath test_float test_gdb test_math test_strtod 1 test altered the execution environment: test_distutils 37 tests skipped: test_aepack test_al test_applesingle test_bsddb test_bsddb185 test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_dl test_gl test_imageop test_imgfile test_kqueue test_linuxaudiodev test_macos test_macostools test_msilib test_ossaudiodev test_scriptpackages test_smtpnet test_socketserver test_startfile test_sunaudiodev test_timeout test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 4 skips unexpected on linux2: test_bsddb test_bsddb3 test_tk test_ttk_guionly make: *** [test] Error 1 *** Drill down to test_strtod error *** $ ./python Python 2.7.3rc1 (2.7:5c52e7c6d868+, Feb 29 2012, 22:10:22) [GCC Intel(R) C++ gcc 4.6 mode] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from test import test_strtod >>> test_strtod.test_main() test_bigcomp (test.test_strtod.StrtodTests) ... FAIL test_boundaries (test.test_strtod.StrtodTests) ... FAIL test_halfway_cases (test.test_strtod.StrtodTests) ... ok test_parsing (test.test_strtod.StrtodTests) ... FAIL test_particular (test.test_strtod.StrtodTests) ... FAIL test_short_halfway_cases (test.test_strtod.StrtodTests) ... ok test_underflow_boundary (test.test_strtod.StrtodTests) ... FAIL ====================================================================== FAIL: test_bigcomp (test.test_strtod.StrtodTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 214, in test_bigcomp self.check_strtod(s) File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 105, in check_strtod "expected {}, got {}".format(s, expected, got)) AssertionError: Incorrectly rounded str->float conversion for 81608e-328: expected 0x0.0000000000002p-1022, got 0x0.0p+0 ====================================================================== FAIL: test_boundaries (test.test_strtod.StrtodTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 191, in test_boundaries self.check_strtod(s) File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 105, in check_strtod "expected {}, got {}".format(s, expected, got)) AssertionError: Incorrectly rounded str->float conversion for 22250738585072002149149e-330: expected 0x0.ffffffffffffep-1022, got 0x0.0p+0 ====================================================================== FAIL: test_parsing (test.test_strtod.StrtodTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 243, in test_parsing self.check_strtod(s) File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 105, in check_strtod "expected {}, got {}".format(s, expected, got)) AssertionError: Incorrectly rounded str->float conversion for -6.E-310: expected -0x0.06e7344a56502p-1022, got -0x0.0p+0 ====================================================================== FAIL: test_particular (test.test_strtod.StrtodTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 393, in test_particular self.check_strtod(s) File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 105, in check_strtod "expected {}, got {}".format(s, expected, got)) AssertionError: Incorrectly rounded str->float conversion for 12579816049008305546974391768996369464963024663104e-357: expected 0x0.90bbd7412d19fp-1022, got 0x0.0p+0 ====================================================================== FAIL: test_underflow_boundary (test.test_strtod.StrtodTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 205, in test_underflow_boundary self.check_strtod(s) File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 105, in check_strtod "expected {}, got {}".format(s, expected, got)) AssertionError: Incorrectly rounded str->float conversion for 24703282292062327208828439643411068618252990130716238221279284125033775363572e-400: expected 0x0.0000000000001p-1022, got 0x0.0p+0 ---------------------------------------------------------------------- Ran 7 tests in 0.280s FAILED (failures=5) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/src/pysrc/cpython/Lib/test/test_strtod.py", line 396, in test_main test_support.run_unittest(StrtodTests) File "/usr/local/src/pysrc/cpython/Lib/test/test_support.py", line 1094, in run_unittest _run_suite(suite) File "/usr/local/src/pysrc/cpython/Lib/test/test_support.py", line 1077, in _run_suite raise TestFailed(err) test.test_support.TestFailed: multiple errors occurred *** Binary size and linked libraries *** ## My Intel build ## $ ls -l ./python && ldd ./python -rwxrwxr-x 1 user user 5.2M 2012-02-29 22:10 ./python linux-vdso.so.1 => (0x00007fffde1ec000) libirc.so => /usr/intel/composer_xe_2011_sp1.9.293/compiler/lib/intel64/libirc.so (0x00007fe5f0f30000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe5f0cde000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe5f0ada000) libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fe5f08d7000) libimf.so => /usr/intel/composer_xe_2011_sp1.9.293/compiler/lib/intel64/libimf.so (0x00007fe5f050b000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe5f0287000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe5f0071000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe5efcd1000) /lib64/ld-linux-x86-64.so.2 (0x00007fe5f107e000) libintlc.so.5 => /usr/intel/composer_xe_2011_sp1.9.293/compiler/lib/intel64/libintlc.so.5 (0x00007fe5efb85000) ## System build ## $ ls -lhH /usr/bin/python && ldd /usr/bin/python -rwxr-xr-x 1 root root 2.7M 2011-10-04 22:26 /usr/bin/python linux-vdso.so.1 => (0x00007fff509ff000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3e339b0000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3e337ab000) libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f3e335a8000) libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f3e33357000) libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f3e32fa7000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f3e32d8f000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3e32b0b000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3e3276b000) /lib64/ld-linux-x86-64.so.2 (0x00007f3e33c03000) *** Conclusion (finally!) *** The Intel Python build looks very promising, but I don't yet trust it to the extent that I'd to go ahead and install it or use it in place of the system build. None of the errors look too alarming though, so I'm confident that I could actually get this to work, with the right help. If someone could help me pass these final tests and compile the ffi64.c module, that'd be amazing! I hope to hear back from you, Kind regards, Alex ps. Sorry how long this email turned out! pps. I'd be happy to write up the fully working solution on a wiki or somewhere, if anyone has any suggestions where?

3 4

Re: [Python-Dev] Compiling Python on Linux with Intel's icc
by Alex Leach March 2, 2012

March 2, 2012

Stefan Krah <stefan at bytereef.org> wrote: > Alex Leach <albl500 at york.ac.uk> wrote: > > I've managed to compile everything in the python distribution except for > > Modules/_ctypes/libffi/src/x86/ffi64.c. > > There is an issue for this: > > http://bugs.python.org/issue4130 Yes, I saw that bug report, but it looked dormant. It is. In 4 years it's only had one post (from you I see), and no proposed fix. The link you posted there is the same link I posted (somewhere) in my previous email... > > After compilation, there's a few tests that are consistently failing, mainly > > involved with floating point precision: test_cmath, test_math and test_float. > > I think you have to compile with "-fp-model strict". > Thanks, I'll give that a go and will report back! > > In general, please submit suspected bugs on http://bugs.python.org/ > (after searching the archives) and post things like speed comparisons on> python-list at python.org. > Thanks again. My only other concern is with distutils, as it doesn't support icc on a Xeon. However, numpy.distutils is almost compatible. I've had to make some mods to the flags in numpy.distutils.intelccompiler and numpy.distutils.fcompiler.intel, but it would be nice if this support was also included in the standard distutils... Can the numpy version be used in place of the standard distutils? Again, there's probably a more proper place to ask... I'll suggest patches for these numpy modules to the numpy devs, but it would be nice if the core python distutils supported icc too. Thanks for your time! Kind regards, Alex

2 1

Add a frozendict builtin type
by Victor Stinner March 2, 2012

March 2, 2012

Rationale ========= A frozendict type is a common request from users and there are various implementations. There are two main Python implementations: * "blacklist": frozendict inheriting from dict and overriding methods to raise an exception when trying to modify the frozendict * "whitelist": frozendict not inheriting from dict and only implement some dict methods, or implement all dict methods but raise exceptions when trying to modify the frozendict The blacklist implementation has a major issue: it is still possible to call write methods of the dict class (e.g. dict.set(my_frozendict, key, value)). The whitelist implementation has an issue: frozendict and dict are not "compatible", dict is not a subclass of frozendict (and frozendict is not a subclass of dict). I propose to add a new frozendict builtin type and make dict type inherits from it. frozendict would not have methods to modify its content and values must be immutable. Constraints =========== * frozendict values must be immutable, as dict keys * frozendict can be used with the C API of the dict object (e.g. PyDict_GetItem) but write methods (e.g. PyDict_SetItem) would fail with a TypeError ("expect dict, got frozendict") * frozendict.__hash__() has to be determinist * frozendict has not the following methods: clear, __delitem__, pop, popitem, setdefault, __setitem__ and update. As tuple/frozenset has less methods than list/set. * issubclass(dict, frozendict) is True, whereas issubclass(frozendict, dict) is False Implementation ============== * Add an hash field to the PyDictObject structure * Make dict inherits from frozendict * frozendict values are checked for immutability property by calling their __hash__ method, with a fast-path for known immutable types (int, float, bytes, str, tuple, frozenset) * frozendict.__hash__ computes hash(frozenset(self.items())) and caches the result is its private hash attribute Attached patch is a work-in-progress implementation. TODO ==== * Add a frozendict abstract base class to collections? * frozendict may not overallocate dictionary buckets? -- Examples of frozendict implementations: http://bob.pythonmac.org/archives/2005/03/04/frozendict/ http://code.activestate.com/recipes/498072-implementing-an-immutable-dictio… http://code.activestate.com/recipes/414283-frozen-dictionaries/ http://corebio.googlecode.com/svn/trunk/apidocs/corebio.utils.frozendict-cl… http://code.google.com/p/lingospot/source/browse/trunk/frozendict/frozendic… http://cmssdt.cern.ch/SDT/doxygen/CMSSW_4_4_2/doc/html/d6/d2f/classfrozendi… See also the recent discussion on python-list: http://mail.python.org/pipermail/python-list/2012-February/1287658.html -- See also the PEP 351. Victor

25 75

Re: [Python-Dev] Compiling Python on Linux with Intel's icc
by Alex Leach March 2, 2012

March 2, 2012

> Éric Araujo <merwok at netwok.org> wrote: > > Could you expand on that? distutils is supposed to support all > unix-like C compilers. Packages that use the numpy distutils can be built with the following options:- $ python setup.py config --compiler=intelem --fcompiler=intelem build -- compiler=intelem install This allows distutils to set the appropriate compile flags. Modules built with the normal distutils always raise warnings about unsupported flags, e.g. -fwrapv. icc needs to calm down a bit on certain floating point arithmetic optimisations, needing some option like '-fp-model strict', as Stefan suggested. '-xHost' is a good flag to use too, allowing icc to detect the CPU type, and use appropriate optimisations. The only way I can build modules now is by using environment variables, e.g:- $ CC=icc CXX=icpc LD=xild AR=xiar python setup.py config build build_ext But this then uses gcc-specific flags when compiling. Cheers, Alex

1 0

Re: [Python-Dev] Compiling Python on Linux with Intel's icc
by Alex Leach March 2, 2012

March 2, 2012

Stefan Krah <stefan at bytereef.org> wrote: > Alex Leach <albl500 at york.ac.uk> wrote: > > I've managed to compile everything in the python distribution except for > > Modules/_ctypes/libffi/src/x86/ffi64.c. > > There is an issue for this: > > http://bugs.python.org/issue4130 Yes, I saw that bug report, but it looked dormant. It is. In 4 years it's only had one post (from you I see), and no proposed fix. The link you posted there is the same link I posted (somewhere) in my previous email... > > After compilation, there's a few tests that are consistently failing, mainly > > involved with floating point precision: test_cmath, test_math and test_float. > > I think you have to compile with "-fp-model strict". > Thanks, I'll give that a go and will report back! > > In general, please submit suspected bugs on http://bugs.python.org/ > (after searching the archives) and post things like speed comparisons on> python-list at python.org. > Thanks again. My only other concern is with distutils, as it doesn't support icc on a Xeon. However, numpy.distutils is almost compatible. I've had to make some mods to the flags in numpy.distutils.intelccompiler and numpy.distutils.fcompiler.intel, but it would be nice if this support was also included in the global distutils... Can the numpy version be used in place of the standard distutils? Again, there's probably a more proper place to ask... I'll suggest patches for these numpy modules to the numpy devs, but it would be nice if the core python distutils supported icc too. Thanks for your time! Kind regards, Alex

2 1

Re: [Python-Dev] Compiling Python on Linux with Intel's icc
by Alex Leach March 2, 2012

March 2, 2012

Stefan Krah <stefan at bytereef.org> wrote: > Alex Leach <albl500 at york.ac.uk> wrote: > > I've managed to compile everything in the python distribution except for > > Modules/_ctypes/libffi/src/x86/ffi64.c. > > There is an issue for this: > > http://bugs.python.org/issue4130 Yes, I saw that bug report, but it looked dormant. It is. In 4 years it's only had one post (from you I see), and no proposed fix. The link you posted there is the same link I posted (somewhere) in my previous email... > > After compilation, there's a few tests that are consistently failing, mainly > > involved with floating point precision: test_cmath, test_math and test_float. > > I think you have to compile with "-fp-model strict". > Thanks, I'll give that a go and will report back! > > In general, please submit suspected bugs on http://bugs.python.org/ > (after searching the archives) and post things like speed comparisons on> python-list at python.org. > Thanks again. My only other concern is with distutils, as it doesn't support icc on a Xeon. However, numpy.distutils is almost compatible. I've had to make some mods to the flags in numpy.distutils.intelccompiler and numpy.distutils.fcompiler.intel, but it would be nice if this support was also included in the standard distutils... Can the numpy version be used in place of the standard distutils? Again, there's probably a more proper place to ask... I'll suggest patches for these numpy modules to the numpy devs, but it would be nice if the core python distutils supported icc too. Thanks for your time! Kind regards, Alex

1 0

PEP 414 - Unicode Literals for Python 3
by Armin Ronacher March 1, 2012

March 1, 2012

Hi, I just uploaded PEP 414 which proposes am optional 'u' prefix for string literals for Python 3. You can read the PEP online: http://www.python.org/dev/peps/pep-0414/ This is a followup to the discussion about this topic here on the mailinglist and on twitter/IRC over the last few weeks. Regards, Armin

32 161

EuroPython 2012: Call for Proposal is Open! [Please spread the word]
by Francesco Pallanti March 1, 2012

March 1, 2012

Hi guys, I'm Francesco and I am writing on behalf of EuroPython Staff (www.europython.eu). We are happy to announce that the Call for Proposals is now officially open! DEADLINE FOR PROPOSALS: MARCH 18TH, 23:59:59 CET For those who have never been at EuroPython (or similar conferences) before, the Call for Proposals is the period in which the organizers ask the community to submit proposals for talks to be held at the conference. Further details about Call for Proposal are online here: http://ep2012.europython.eu/call-for-proposals/ EuroPython is a conference run by the community for the community: the vast majority of talks that are presented at the conference will be proposed, prepared and given by members of the Python community itself. And not only that: the process that selects the best talks among all the proposals will also be public and fully driven by the community: it's called Community Voting, and will begin right after the Call for Proposals ends. CFP: Talks, Hands-On Trainings and Posters ------------------------------------------ We're looking for proposals on every aspect of Python: programming from novice to advanced levels, applications and frameworks, or how you have been involved in introducing Python into your organisation. There are three different kind of contribution that you can present at EuroPython: - Regular talk. These are standard "talk with slides", allocated in slots of 45, 60 or 90 minutes, depending on your preference and scheduling constraints. A Q&A session is held at the end of the talk. - Hands-on training. These are advanced training sessions for a smaller audience (10-20 people), to dive into the subject with all details. These sessions are 4-hours long, and the audience will be strongly encouraged to bring a laptop to experiment. They should be prepared with less slides and more source code. - Posters. Posters are a graphical way to describe a project or a technology, printed in large format; posters are exhibited at the conference, can be read at any time by participants, and can be discussed face to face with their authors during the poster session. We will take care of printing the posters too, so don't worry about logistics. More details about Call for Proposal are online here: http://ep2012.europython.eu/call-for-proposals/ Don't wait for the last day --------------------------- If possible, please avoid submitting your proposals on the last day. It might sound a strange request, but last year about 80% of the proposals were submitted in the last 72 hours. This creates a few problems for organizers because we can't have a good picture of the size of the conference until that day. Remember that proposals are fully editable at any time, even after the Call for Proposals ends. You just need to login on the website, go to the proposal page (linked from your profile page), and click the Edit button. First-time speakers are especially welcome; EuroPython is a community conference and we are eager to hear about your experience. If you have friends or colleagues who have something valuable to contribute, twist their arms to tell us about it! We are a conference run by the community for the community. Please help to spread the word by distributing this announcement to colleagues, mailing lists, your blog, Web site, and through your social networking connections. All the best, -- Francesco Pallanti - fpallanti(a)develer.com Develer S.r.l. - http://www.develer.com/ .software .hardware .innovation Tel.: +39 055 3984627 - ext.: 215

1 0

State of PEP-3118 (memoryview part)
by Stefan Krah March 1, 2012

March 1, 2012

State of PEP-3118 (memoryview part) Hello, In Python 3.3 most issues with the memoryview object have been fixed in a recent commit (3f9b3b6f7ff0). Many features have been added, see: http://docs.python.org/dev/whatsnew/3.3.html The underlying problems with memoryview were intricate and required a long discussion (issue #10181) that led to a complete rewrite of memoryobject.c. We have several options with regard to 2.7 and 3.2: 1) Won't fix. 2) Backport the changes and disable as much of the new functionality as possible. 3) Backport all of it (this would be the least amount of work and could be done relatively quickly). 4) Nick suggested another option: put a module with the new functionality on PyPI. This would be quite a bit of work, and personally I don't have time for that. Options 2) and 3) would ideally entail one backwards incompatible bugfix: In 2.7 and 3.2 assignment to a memoryview with format 'B' rejects integers but accepts byte objects, but according to the struct syntax mandated by the PEP it should be the other way round. It would be nice to get some opinions and ideas, especially of course from the release managers. Stefan Krah

7 12