Mailman 3 February 2009 - Python-Dev

Re: [Python-Dev] multiprocessing not compatible with functional.partial
by Hrvoje Niksic 12 Feb '09

12 Feb '09

Calvin Spealman wrote: > I don't think it would be unreasonable to consider either 1) making > functools.partial picklable (I don't know how feasible this is) It's not only feasible, but quite easy and, I think, useful. A "partial" instance is a simple triplet of (function, args, kwds), and it can be pickled as such. For example: >>> import copy_reg, functools >>> def _reconstruct_partial(f, args, kwds): ... return functools.partial(f, *args, **(kwds or {})) ... >>> def _reduce_partial(p): ... return _reconstruct_partial, (p.func, p.args, p.keywords) ... >>> copy_reg.pickle(functools.partial, _reduce_partial) Test: >>> import operator, cPickle as cp >>> p = functools.partial(operator.add, 3) >>> p(10) 13 >>> cp.dumps(p) 'c__main__\n_reconstruct_partial\np1\n(coperator\nadd\np2\n(I3\ntp3\nNtRp4\n.' >>> p2 = cp.loads(_) >>> p2(10) 13 Iedally this should be implemented in the functools.partial object itself.

3 3

multiprocessing not compatible with functional.partial
by Neal Becker 12 Feb '09

12 Feb '09

If the argument to pool.map (f, args) is f = functional.partial (my_func, some_keyword_arg=whatever) I get: Traceback (most recent call last): File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- self.run() File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- self._target(*self._args, **self._kwargs) File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- task = get() File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- return recv() TypeError: type 'partial' takes at least one argument

4 3

Python 3.0.1 planned for this Friday, Feb 13
by Barry Warsaw 12 Feb '09

12 Feb '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 A quick reminder that I am planning to release Python 3.0.1 this Friday, February 13. Thanks to y'all's hard work, we have no showstopper bugs at the moment. The 3.0 buildbots look mostly clean and green. If we can keep it this way for about 24 hours, I will tag the branch at about 2300 UTC Thursday February 12 so that Martin can build the Windows installers on his Friday morning. Then Friday evening at about 2300 UTC, I will spin the tarballs and make the release. Cheers, Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZNhLXEjvBPtnXfVAQKxoQP9HGUioYy3i0Z4glQfRevwJyiIvYqA1cF1 DKHpCn/8eJ/QXJ32Sfh/H2cASpDwkNNQLHrBOnzOPbe4M8ZJisg8bC27g+fTxckZ 5+yUJUTtRxk7/nDwWsS/DAWZ17VT9z8YN43XO2gGhcXw2YYl1Mi7VWvuZed9ojVL D7Gwy10S+s4= =RYa0 -----END PGP SIGNATURE-----

3 3

test_tk failing
by Raymond Hettinger 12 Feb '09

12 Feb '09

C:\py27>py27 lib\test\regrtest.py -uall test_tk test_tk test test_tk failed -- Traceback (most recent call last): File "c:\py27\lib\lib-tk\test\test_tkinter\test_text.py", line 32, in test_search self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2') AssertionError: <textindex object at 0140D708> != '1.2' 1 test failed: test_tk [35724 refs] C:\py31>py31 lib\test\regrtest.py -uall test_tk test_tk test test_tk failed -- Traceback (most recent call last): File "c:\py31\lib\tkinter\test\test_tkinter\test_text.py", line 32, in test_search self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2') AssertionError: <textindex object at 0x010A33F0> != '1.2' 1 test failed: test_tk [73837 refs]

2 1

python seg faults
by Sushant Sinha 11 Feb '09

11 Feb '09

I am running Python 2.5.4 compiled with gcc (Gentoo 4.3.2-r2 p1.5, pie-10.1.5) 4.3.2 My program uses python interface to two C modules namely libnids (pynids) and adns (adns-python). My program is written all in python. When I run my program it aborts. Here is the stack when that happens: #0 0x00007f6cfdb79205 in raise () from /lib64/libc.so.6 #1 0x00007f6cfdb7a723 in abort () from /lib64/libc.so.6 #2 0x00007f6cfdbb4cf8 in ?? () from /lib64/libc.so.6 #3 0x00007f6cfdbba468 in ?? () from /lib64/libc.so.6 #4 0x00007f6cfdbbbfa6 in free () from /lib64/libc.so.6 #5 0x00007f6cfe79dff2 in ?? () from /usr/lib/libpython2.5.so.1.0 #6 0x00007f6cfe7fd736 in PyEval_EvalFrameEx () from /usr/lib/libpython2.5.so.1.0 #7 0x00007f6cfe7fde7d in PyEval_EvalCodeEx () from /usr/lib/libpython2.5.so.1.0 #8 0x00007f6cfe79f5ce in ?? () from /usr/lib/libpython2.5.so.1.0 #9 0x00007f6cfe781e27 in PyObject_Call () from /usr/lib/libpython2.5.so.1.0 #10 0x00007f6cfe782187 in ?? () from /usr/lib/libpython2.5.so.1.0 #11 0x00007f6cfe7826c8 in PyObject_CallFunction () from /usr/lib/libpython2.5.so.1.0 #12 0x00007f6cfd939b41 in callTcpFunc () from /usr/lib64/python2.5/site-packages/nidsmodule.so #13 0x00007f6cfd93ebae in process_tcp (data=0x1a2f930 "E", skblen=248) at tcp.c:805 #14 0x00007f6cfd93bd7a in gen_ip_proc (data=0x1a2f930 "E", skblen=248) at libnids.c:436 #15 0x00007f6cfd93bbd8 in gen_ip_frag_proc (data=0x1a2f930 "E", len=46) at libnids.c:388 #16 0x00007f6cfd93b6a3 in call_ip_frag_procs (data=0x1a2f930, caplen=46) at libnids.c:198 #17 0x00007f6cfd93ba05 in nids_pcap_handler (par=0x0, hdr=0x7fff06cc3500, data=0x1a2f922 "") at libnids.c:340 #18 0x00007f6cfd7195d9 in pcap_offline_read () from /usr/lib/libpcap.so.1 #19 0x00007f6cfd93c809 in nids_dispatch (cnt=1) at libnids.c:743 #20 0x00007f6cfd939345 in nids_dispatch_exc () from /usr/lib64/python2.5/site-packages/nidsmodule.so #21 0x00007f6cfd93a980 in pynids_run () from /usr/lib64/python2.5/site-packages/nidsmodule.so #22 0x00007f6cfe7fcacd in PyEval_EvalFrameEx () from /usr/lib/libpython2.5.so.1.0 #23 0x00007f6cfe7fde7d in PyEval_EvalCodeEx () from /usr/lib/libpython2.5.so.1.0 #24 0x00007f6cfe7fe062 in PyEval_EvalCode () from /usr/lib/libpython2.5.so.1.0 #25 0x00007f6cfe818601 in ?? () from /usr/lib/libpython2.5.so.1.0 #26 0x00007f6cfe8186d6 in PyRun_FileExFlags () from /usr/lib/libpython2.5.so.1.0 #27 0x00007f6cfe819c6d in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.5.so.1.0 #28 0x00007f6cfe82337a in Py_Main () from /usr/lib/libpython2.5.so.1.0 #29 0x00007f6cfdb655c6 in __libc_start_main () from /lib64/libc.so.6 #30 0x0000000000400789 in _start () When I run under valgrind I see a whole bunch of memory errors. Two of them are show below. ==26882== Thread 1: ==26882== Invalid free() / delete / delete[] ==26882== at 0x4C239FF: free (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so) ==26882== by 0x4E84FF1: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4735: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4E7C: PyEval_EvalCodeEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E865CD: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E68E26: PyObject_Call (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E69186: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E696C7: PyObject_CallFunction (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x618FB40: callTcpFunc (in /usr/lib64/python2.5/site-packages/nidsmodule.so) ==26882== by 0x6194BAD: process_tcp (tcp.c:805) ==26882== by 0x6191D79: gen_ip_proc (libnids.c:436) ==26882== by 0x6191BD7: gen_ip_frag_proc (libnids.c:388) ==26882== Address 0x9195230 is 168,568 bytes inside a block of size 262,144 alloc'd ==26882== at 0x4C24CFE: malloc (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so) ==26882== by 0x4E9BEE8: PyObject_Malloc (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4F0B468: _PyObject_GC_Malloc (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4F0B55D: _PyObject_GC_NewVar (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EAB313: PyTuple_New (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EF9B28: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EFA0A0: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EF9B75: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EFA08A: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EFA755: PyMarshal_ReadObjectFromString (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EFC272: PyMarshal_ReadLastObjectFromFile (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EF453D: (within /usr/lib64/libpython2.5.so.1.0) ==26882== Invalid read of size 4 ==26882== at 0x4E9C20B: PyObject_Realloc (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EA622E: _PyString_Resize (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EDE458: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE3128: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4E7C: PyEval_EvalCodeEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E865CD: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E68E26: PyObject_Call (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E69186: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E696C7: PyObject_CallFunction (in /usr/lib64/libpython2.5.so.1.0) ==26882== Address 0xb380020 is 280 bytes inside a block of size 1,104 free'd ==26882== at 0x4C239FF: free (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so) ==26882== by 0x4F14E2B: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4F1A09A: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE3ACC: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4E7C: PyEval_EvalCodeEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E865CD: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E68E26: PyObject_Call (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E69186: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E696C7: PyObject_CallFunction (in /usr/lib64/libpython2.5.so.1.0) So I am not sure where the error is. Any clue on where the bug possibly may be: adns-python, pynids or python? Or how I should I go about debugging this? -Sushant.

4 3

Python-acceleration instructions on ARM
by Benjamin Schwartz 11 Feb '09

11 Feb '09

Dear Python developers, Introduction: I am writing from the perspective of Sugar Labs [1], which produces Sugar, a free software project written almost entirely in Python. Sugar is designed to run on small, resource-constrained computers. So far those computers have been mostly x86, but it seems likely to me that we will soon want to run on ARM as well, with the new wave of small ARM laptops [2]. These laptops are likely to run on some variant of the ARM Cortex-A8 CPU core. The Cortex-A8 chips all contain a set of commands known as ThumbEE or Jazelle RCT (Runtime Compilation Target) [3]. According to ARM [4]: """Jazelle RCT can be used to significantly reduce the code bloat associated with AOT and JIT compilation, making AOT technology viable on mass-market devices. It can also be used to support execution environments beyond Java, such as Microsoft .NET Compact Framework, Python and others.""" """Jazelle RCT provides an excellent target for any run-time compilation technology, including JIT and AOT for .NET MSIL, Python and Perl as well as Java. ARM is working with leading software providers to enable solutions ready for market with Jazelle RCT.""" The Jazelle RCT system consists of 12 assembly instructions, documented at [5] and [6]. Question: ARM is specifically claiming that these instructions can be used to accelerate Python interpretation. Is there any interpreter code, in CPython or elsewhere, that uses ThumbEE mode? Is there anyone working on this? What would the process be to incorporate the use of ThumbEE instructions into CPython? The whitepaper mentions the Parrot interpreter specifically, but I cannot find any indication that anyone is actually working on Jazelle RCT support in Parrot. Thank you, Ben Schwartz [1] http://sugarlabs.org/go/Main_Page [2] http://www.engadget.com/2009/01/09/pegatron-and-freescale-team-for-low-powe… [3] http://en.wikipedia.org/wiki/ARM_architecture#Thumb_Execution_Environment_.… [4] http://www.arm.com/products/multimedia/java/jazelle_architecture.html [5] http://infocenter.arm.com/help/topic/com.arm.doc.dui0379a/CIHBCDGA.html [6] http://www.arm.com/pdfs/JazelleRCTWhitePaper_final1-0_.pdf -- View this message in context: http://www.nabble.com/Python-acceleration-instructions-on-ARM-tp21947336p21… Sent from the Python - python-dev mailing list archive at Nabble.com.

7 10

Re: [Python-Dev] IDLE reading IDLESTARTUP or PYTHONSTARTUP on restart
by Raghuram Devarakonda 11 Feb '09

11 Feb '09

On Wed, Feb 11, 2009 at 1:45 PM, Mitchell L Model <MLMLists(a)comcast.net> wrote: > I have a patch for IDLE, but I've never submitted a patch before and not > quite sure of the procedure, despite reading the guidelines at > http://www.python.org/dev/patches/. But I'll be brave and persevere. The > question I have at the moment is should I only submit the patch vs. 3.1 or > also for 3.0? 2.7? 2.6? 2.5? The guidelines listed above clearly state that patches should be sent relative to the current SVN tree. That would be 2.7, I guess. In addition, I would suggest uploading the patch to http://codereview.appspot.com which makes reviewing simpler. Perhaps, the guidelines page can mention this point. > Guidance on all of these would be greatly appreciated. The patch itself > disturbs things in a half-dozen places, but in very minimal ways, so it's The recommended way is to create a tracker issue (with or without patch) and send a mail here with the issue link and a brief description. That will allow interested people to check the issue and add comments. Thanks, Raghu

1 0

Daily documentation builds
by Raymond Hettinger 11 Feb '09

11 Feb '09

The online development docs stopped their nightly rebuilds four days ago. Raymond

3 3

IDLE reading IDLESTARTUP or PYTHONSTARTUP on restart
by Mitchell L Model 11 Feb '09

11 Feb '09

I have a patch for IDLE, but I've never submitted a patch before and not quite sure of the procedure, despite reading the guidelines at http://www.python.org/dev/patches/. But I'll be brave and persevere. The question I have at the moment is should I only submit the patch vs. 3.1 or also for 3.0? 2.7? 2.6? 2.5? The main thing the patch does is: modify the subprocess restart procedure so that it reloads whatever file, if any, was loaded when IDLE first started and looked for IDLESTARTUP then PYTHONSTARTUP environment variables. In addition: a -q option is added for starting IDLE on the command line to mean "quiet", as with Emacs, e.g., to suppress loading of IDLESTARTUP or PYTHONSTARTUP The former effect of -s would now be the default, which is desirable so double-clicking an IDLE icon to start it will cause the startup file to run. -s is changed to take an argument that is an alternate startup file to use I am a bit concerned about changing -s to have a different meaning. Perhaps it's better to leave -s as an option that is simplhy superfluous and use a different letter for the alternate startup. Guidance on all of these would be greatly appreciated. The patch itself disturbs things in a half-dozen places, but in very minimal ways, so it's not quite as easy as saying "add these three lines over here". (It was that simple until I started thinking about whether the restart should search for IDLESTARTUP/PYTHONSTARTUP again, and perhaps encounter a different file than IDLE did on startup [I think it shouldn't, but it's not a big deal either way] and if shouldn't, then whatever path was originally found has to be stored somewhere and used later. That led to the thinking behind the changes to the startup switches.

1 0

Re: [Python-Dev] Expression optimizations
by Cesare Di Mauro 10 Feb '09

10 Feb '09

It's bizarre enough, since I have got a different result (with Python 2.6.1, 32 bit): >>> x = 1e66666 >>> y = x/x >>> x inf >>> y nan >>> cmp(y, y) 0 >>> cmp(x/x, x/x) 1 :D Cesare On Mar, Feb 10, 2009 10:02PM, Dino Viehland wrote: > And slightly unrelated, but just showing how bizarre floats are: > >>>> x = 1e66666 >>>> y = x/x >>>> cmp(y, y) > 0 >>>> cmp(x/x, x/x) > -1 > > Yeah object identity checks! > > From: python-dev-bounces+dinov=microsoft.com(a)python.org > [mailto:python-dev-bounces+dinov=microsoft.com@python.org] On Behalf Of > Daniel Stutzbach > Sent: Tuesday, February 10, 2009 12:43 PM > To: cesare.dimauro(a)a-tono.com > Cc: Python-Dev > Subject: Re: [Python-Dev] Expression optimizations > > On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro > <cesare.dimauro(a)a-tono.com> wrote: > OK, so I can make assumptions only for built-in types. > > Yes, but even there you have to be careful of odd corner-cases, such as: > >>>> nan = float('nan') >>>> nan < nan > False >>>> nan >= nan > False > -- > Daniel Stutzbach, Ph.D. > President, Stutzbach Enterprises, LLC >

2 1