Mailman 3 October 2006 - Python-Dev

Re: [Python-Dev] Python 2.5 performance
by Kristján V. Jónsson 16 Oct '06

16 Oct '06

Well, it ought to be possible. I can turn off the instrumentation on the other modules, and see what happens. K > -----Original Message----- > From: Giovanni Bajo [mailto:rasky@develer.com] > Sent: 12. október 2006 20:30 > To: Kristján V. Jónsson > Cc: python-dev(a)python.org > Subject: Re: Python 2.5 performance > > Kristján V. Jónsson wrote: > > > This is an improvement of another 3.5 %. > > In all, we have a performance increase of more than 10%. > > Granted, this is from a single set of runs, but I think we should > > start considering to make PCBuild8 a "supported" build. > > Kristján, I wonder if the performance improvement comes from > ceval.c only (or maybe a few other selected files). Is it > possible to somehow link the PGO-optimized ceval.obj into the > VS2003 project? > -- > Giovanni Bajo > >

1 0

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser 16 Oct '06

16 Oct '06

Patch / Bug Summary ___________________ Patches : 431 open ( +3) / 3425 closed ( +8) / 3856 total (+11) Bugs : 916 open (-23) / 6273 closed (+44) / 7189 total (+21) RFE : 244 open ( +4) / 240 closed ( +1) / 484 total ( +5) New / Reopened Patches ______________________ typo in PC/_msi.c (2006-10-07) CLOSED http://python.org/sf/1572724 opened by jose nazario Fix for segfault in ISO 2022 codecs (2006-10-07) CLOSED http://python.org/sf/1572832 opened by Ray Chason let quit and exit really exit (2006-10-09) CLOSED http://python.org/sf/1573835 opened by Gerrit Holl urllib2 - Fix line breaks in authorization headers (2006-10-09) http://python.org/sf/1574068 opened by Scott Dial Add %var% support to ntpath.expandvars (2006-10-09) http://python.org/sf/1574252 opened by Chip Norkus Mailbox will lock properly after flush() (2006-10-11) http://python.org/sf/1575506 opened by Philippe Gauthier Support spawnvp[e] + use native execvp[e] on win32 (2006-10-12) http://python.org/sf/1576120 opened by Snaury os.utime acess denied with directories on win32 (2006-10-12) CLOSED http://python.org/sf/1576166 opened by Snaury os.execvp[e] on win32 fails for current directory (2006-10-13) http://python.org/sf/1576313 opened by Snaury Fix VC6 build, remove redundant files for VC7 build (2006-10-14) CLOSED http://python.org/sf/1576954 opened by Larry Hastings Fix VC6 build, remove redundant files for VC7 build (2006-10-14) CLOSED http://python.org/sf/1577078 opened by Larry Hastings Add _ctypes, _ctypes_test, and _elementtree to VC6 build (2006-10-15) CLOSED http://python.org/sf/1577551 opened by Larry Hastings newline in -DSVNVERSION=\"`LANG=C svnversion .`\" (2006-10-15) CLOSED http://python.org/sf/1577756 opened by Daniel Stränger Patches Closed ______________ typo in PC/_msi.c (2006-10-07) http://python.org/sf/1572724 closed by gbrandl Fix for segfault in ISO 2022 codecs (2006-10-08) http://python.org/sf/1572832 closed by perky fix crash with continue in nested try/finally (2006-08-18) http://python.org/sf/1542451 closed by gbrandl let quit and exit really exit (2006-10-09) http://python.org/sf/1573835 closed by mwh Fix for Lib/test/crashers/gc_inspection.py (2006-07-04) http://python.org/sf/1517042 closed by gbrandl os.utime acess denied with directories on win32 (2006-10-12) http://python.org/sf/1576166 closed by loewis Fix VC6 build, remove redundant files for VC7 build (2006-10-14) http://python.org/sf/1576954 closed by loewis Fix VC6 build, remove redundant files for VC7 build (2006-10-14) http://python.org/sf/1577078 deleted by lhastings Add _ctypes, _ctypes_test, and _elementtree to VC6 build (2006-10-15) http://python.org/sf/1577551 closed by loewis newline in -DSVNVERSION=\"`LANG=C svnversion .`\" (2006-10-15) http://python.org/sf/1577756 deleted by schmaller New / Reopened Bugs ___________________ cElementTree.SubElement doesn't recognize keyword "attrib" (2006-10-07) CLOSED http://python.org/sf/1572710 opened by Mark Stephens import org.python.core imports local org.py (2006-10-08) CLOSED http://python.org/sf/1573180 opened by E.-O. Le Bigot ctypes unit test fails (test_macholib.py) under MacOS 10.4.7 (2006-08-21) CLOSED http://python.org/sf/1544102 reopened by ronaldoussoren struct module doesn't use weakref for cache (2006-10-08) CLOSED http://python.org/sf/1573394 opened by Mark Flacy sqlite3 documentation on rowcount is contradictory (2006-10-10) http://python.org/sf/1573854 opened by Seo Sanghyeon if(status = ERROR_MORE_DATA) (2006-10-09) CLOSED http://python.org/sf/1573928 opened by Helmut Grohne WSGI, cgi.FieldStorage incompatibility (2006-10-09) http://python.org/sf/1573931 opened by Michael Kerrin isinstance swallows exceptions (2006-10-09) http://python.org/sf/1574217 opened by Brian Harring os.popen with os.close gives error message (2006-10-10) http://python.org/sf/1574310 opened by dtrosset Error with callback function and as_parameter with NumPy ndp (2006-10-10) http://python.org/sf/1574584 opened by Albert Strasheim ctypes: Pointer-to-pointer unchanged in callback (2006-10-10) http://python.org/sf/1574588 opened by Albert Strasheim ctypes: Returning c_void_p from callback doesn't work (2006-10-10) http://python.org/sf/1574593 opened by Albert Strasheim Request wave support > 16 bit samples (2006-10-11) http://python.org/sf/1575020 opened by Murray Lang isSequenceType returns True for dict subclasses (<> 2.3) (2006-10-11) http://python.org/sf/1575169 opened by Martin Gfeller typo: section 2.1 -> property (2006-10-12) CLOSED http://python.org/sf/1575746 opened by Antoine De Groote Missing notice on environment setting LD_LIBRARY_PATH (2006-10-12) CLOSED http://python.org/sf/1575803 opened by Anastasios Hatzis from_param and _as_parameter_ truncating 64-bit value (2006-10-12) http://python.org/sf/1575945 opened by Albert Strasheim str(WindowsError) wrong (2006-10-12) http://python.org/sf/1576174 opened by Thomas Heller ConfigParser: whitespace leading comment lines (2006-10-12) http://python.org/sf/1576208 opened by gregwillden functools.wraps fails on builtins (2006-10-12) http://python.org/sf/1576241 opened by kajiuma Example typo in section 4 of 'Installing Python Modules' (2006-10-13) http://python.org/sf/1576348 opened by ytrewq1 enable-shared .dso location (2006-10-12) CLOSED http://python.org/sf/1576394 opened by Mike Klaas cStringIO misbehaving with unicode (2006-10-13) CLOSED http://python.org/sf/1576443 opened by Yang Zhang ftplib doesn't follow standard (2006-10-13) http://python.org/sf/1576598 opened by Denis S. Otkidach dict keyerror formatting and tuples (2006-10-13) http://python.org/sf/1576657 opened by M.-A. Lemburg potential buffer overflow in complexobject.c (2006-10-13) http://python.org/sf/1576861 opened by Jochen Voss GetFileAttributesExA and Win95 (2006-09-29) CLOSED http://python.org/sf/1567666 reopened by giomach Bugs Closed ___________ csv "dialect = 'excel-tab'" to use excel_tab (2006-10-06) http://python.org/sf/1572471 closed by montanaro cElementTree.SubElement doesn't recognize keyword "attrib" (2006-10-07) http://python.org/sf/1572710 closed by effbot tabs missing in idle options configure (2006-09-28) http://python.org/sf/1567450 closed by kbk IDLE doesn't load - apparently without firewall problems (2006-09-22) http://python.org/sf/1563630 closed by kbk Let assign to as raise SyntaxWarning as well (2003-02-23) http://python.org/sf/691733 closed by nnorwitz cvs update warnings (2003-07-02) http://python.org/sf/764447 closed by nnorwitz Minor floatobject.c bug (2003-08-15) http://python.org/sf/789159 closed by nnorwitz another threads+readline+signals nasty (2004-06-11) http://python.org/sf/971213 closed by nnorwitz init_types (2006-09-30) http://python.org/sf/1568243 closed by gbrandl import org.python.core imports local org.py (2006-10-08) http://python.org/sf/1573180 closed by gbrandl PGIRelease linkage fails on pgodb80.dll (2006-10-02) http://python.org/sf/1569517 closed by krisvale missing _typesmodule.c,Visual Studio 2005 pythoncore.vcproj (2006-09-29) http://python.org/sf/1567910 closed by krisvale ctypes unit test fails (test_macholib.py) under MacOS 10.4.7 (2006-08-21) http://python.org/sf/1544102 closed by ronaldoussoren Tutorial: incorrect info about package importing and mac (2006-09-17) http://python.org/sf/1560114 closed by gbrandl struct module doesn't use weakref for cache (2006-10-08) http://python.org/sf/1573394 closed by etrepum setup() keyword have to be list (doesn't work with tuple) (2006-08-23) http://python.org/sf/1545341 closed by akuchling if(status = ERROR_MORE_DATA) (2006-10-09) http://python.org/sf/1573928 closed by gbrandl os.stat() subsecond file mode time is incorrect on Windows (2006-09-25) http://python.org/sf/1565150 closed by loewis 2.6 changes stomp on 2.5 docs (2006-09-23) http://python.org/sf/1564039 closed by sf-robot Cannot use high-numbered sockets in 2.4.3 (2006-05-24) http://python.org/sf/1494314 closed by anthonybaxter typo: section 2.1 -> property (2006-10-12) http://python.org/sf/1575746 closed by gbrandl -Qnew switch doesn't work (2003-09-26) http://python.org/sf/813342 closed by gbrandl sets missing from standard types list in ref (2006-09-26) http://python.org/sf/1565919 closed by gbrandl make plistlib.py available in every install (2006-09-25) http://python.org/sf/1565129 closed by gbrandl inspect module and class startlineno (2006-09-01) http://python.org/sf/1550524 closed by gbrandl Pdb parser bug (2006-08-30) http://python.org/sf/1549574 closed by gbrandl Missing notice on environment setting LD_LIBRARY_PATH (2006-10-12) http://python.org/sf/1575803 closed by loewis shlex (or perhaps cStringIO) and unicode strings (2006-08-29) http://python.org/sf/1548891 closed by gbrandl Build of 2.4.3 on fedora core 5 fails to find asm/msr.h (2006-09-03) http://python.org/sf/1551238 closed by gbrandl urlparse.urljoin odd behaviour (2006-08-25) http://python.org/sf/1546628 closed by gbrandl inconsistent treatment of NULs in int() (2006-08-23) http://python.org/sf/1545497 closed by gbrandl Move fpectl elsewhere in library reference (2006-09-11) http://python.org/sf/1556261 closed by gbrandl Fix Lib/test/test___all__.py (2006-08-22) http://python.org/sf/1544295 closed by gbrandl smeared title when installing (2004-10-18) http://python.org/sf/1049615 closed by gbrandl nit for builtin sum doc (2005-09-07) http://python.org/sf/1283491 closed by gbrandl site-packages & build-dir python (2002-07-25) http://python.org/sf/586700 closed by gbrandl __name__ doesn't show up in dir() of class (2006-08-03) http://python.org/sf/1534014 closed by gbrandl Interpreter crash: filter() + gc.get_referrers() (2006-07-05) http://python.org/sf/1517663 closed by gbrandl Better/faster implementation of os.path.basename/dirname (2006-09-17) http://python.org/sf/1560179 closed by gbrandl enable-shared .dso location (2006-10-13) http://python.org/sf/1576394 closed by loewis cStringIO misbehaving with unicode (2006-10-13) http://python.org/sf/1576443 closed by gbrandl GetFileAttributesExA and Win95 (2006-09-29) http://python.org/sf/1567666 closed by loewis site-packages isn't created before install_egg_info (2006-09-27) http://python.org/sf/1566719 closed by sf-robot New / Reopened RFE __________________ release GIL while doing I/O operations in the mmap module (2006-10-08) http://python.org/sf/1572968 opened by Lukas Lalinsky RFE Closed __________ Print identical floats consistently (2006-08-05) http://python.org/sf/1534942 closed by gbrandl

1 0

os.utime on directories: bug fix or new feature?
by "Martin v. Löwis" 15 Oct '06

15 Oct '06

In Python 2.5.0 and earlier, it is not possible to modify the time stamps of a directory (mtime and atime) on Windows. The reason is that you cannot "open" (CreateFile) a directory. On W9x, it isn't possible, period. On WNT+, it's possible if you pass FILE_FLAG_BACKUP_SEMANTICS to CreateFile. I just applied patch #1576166 to the trunk which does that. Should I backport the patch to 2.5, as it is a bug that you can modify the time stamps of regular files but not directories? Or should I not backport as it is a new feature that you can now adjust the time stamps of a directory, and couldn't before? Anthony, can you please pronounce? Regards, Martin

3 2

Cloning threading.py using proccesses
by Richard Oudkerk 15 Oct '06

15 Oct '06

I am not sure how sensible the idea is, but I have had a first stab at writing a module processing.py which is a near clone of threading.py but uses processes and sockets for communication. (It is one way of avoiding the GIL.) I have tested it on unix and windows and it seem to work pretty well. (Getting round the lack of os.fork on windows is a bit awkward.) There is also another module dummy_processing.py which has the same api but is just a wrapper round threading.py. Queues, Locks, RLocks, Conditions, Semaphores and some other shared objects are implemented. People are welcome to try out the tests in test_processing.py contained in the zipfile. More information is included in the README file. As a quick example, the code . from processing import Process, Queue, ObjectManager . . def f(token): . q = proxy(token) . for i in range(10): . q.put(i*i) . q.put('STOP') . . if __name__ == '__main__': . manager = ObjectManager() . token = manager.new(Queue) . queue = proxy(token) . . t = Process(target=f, args=[token]) . t.start() . . result = None . while result != 'STOP': . result = queue.get() . print result . . t.join() is not very different from the normal threaded equivalent . from threading import Thread . from Queue import Queue . . def f(q): . for i in range(10): . q.put(i*i) . q.put('STOP') . . if __name__ == '__main__': . queue = Queue() . . t = Thread(target=f, args=[queue]) . t.start() . . result = None . while result != 'STOP': . result = queue.get() . print result . . t.join() Richard

10 20

VC6 support on release25-maint
by ocean 15 Oct '06

15 Oct '06

Hello. I noticed VisualC++6 support came back. I'm glad with that, but still it seems incomplete. (for example, _sqlite3 support) Maybe does this patch help process? On my machine, testcases other than distutils runs fine. http://sourceforge.net/tracker/?func=detail&aid=1457736&group_id=5470&atid=…

2 1

ConfigParser: whitespace leading comment lines
by Greg Willden 14 Oct '06

14 Oct '06

Hello all, I'd like to propose the following change to ConfigParser.py. I won't call it a bug-fix because I don't know the relevant standards. This change will enable multiline comments as follows: [section] item=value ;first of multiline comment ;second of multiline comment Right now the behaviour is In [19]: cfg.get('section','item') Out[19]: 'value\n;second of multiline comment' It's a one-line change. RawConfigParser._read lines 434-437 # comment or blank line? - if line.strip() == '' or line[0] in '#;': + if line.strip() == '' or line.strip()[0] in '#;': continue Regards, Greg Willden (Not a member of python-dev) -- Linux. Because rebooting is for adding hardware.

2 1

Plea to distribute debugging lib
by David Abrahams 14 Oct '06

14 Oct '06

For years, Boost.Python has been doing some hacks to work around the fact that a Windows Python distro doesn't include the debug build of the library. http://www.boost.org/libs/python/doc/building.html#variants explains. We wanted to make it reasonably convenient for Windows developers (and our distributed testers) to work with a debug build of the Boost.Python library and of their own code. Having to download the Python source and build the debug DLL was deemed unacceptable. Well, those hacks have run out of road. VC++8 now detects that some of its headers have been #included with _DEBUG and some without, and it will refuse to build anything when it does. We have several new hacks to work around that detection, and I think we _might_ be able to get away with them for one more release. But it's really time to do it right. MS is recommending that we (Boost) start distributing a debug build of the Python DLL with Boost, but Boost really seems like the wrong place to host such a thing. Is there any way Python.org can make a debug build more accessible? Thanks, Dave -- Dave Abrahams Boost Consulting www.boost-consulting.com

8 17

Re: [Python-Dev] Cloning threading.py using proccesses
by Chetan Pandya 13 Oct '06

13 Oct '06

I just got around to reading the messages. When I first saw this, I thought this is so that the processes that need to share and work on shared objects. That is where the locks are required. However, all shread objects are managed by the object manager and thus all such operations are in effect sequential, even acquires on different locks. Thus other shared objects in the object manager will actually not require any (additional) synchronization. Of course, the argument here is that it is still possible to use that code. Cleanup of shared objects seems to be another thing to look out for. This is a problem that subprocesses seem to avoid and has been already suggested. -Chetan On 10/11/06, python-dev-request(a)python.org <python-dev-request(a)python.org> wrote: > > Message: 5 > Date: Wed, 11 Oct 2006 10:23:40 +0200 > From: "M.-A. Lemburg" <mal(a)egenix.com> > Subject: Re: [Python-Dev] Cloning threading.py using proccesses > To: Josiah Carlson <jcarlson(a)uci.edu> > Cc: python-dev(a)python.org > Message-ID: <452CAA0C.6030306(a)egenix.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Josiah Carlson wrote: > > Fredrik Lundh <fredrik(a)pythonware.com> wrote: > >> Josiah Carlson wrote: > >> > >>> Presumably with this library you have created, you have also written a > >>> fast object encoder/decoder (like marshal or pickle). If it isn't any > >>> faster than cPickle or marshal, then users may bypass the module and > opt > >>> for fork/etc. + XML-RPC > >> XML-RPC isn't close to marshal and cPickle in performance, though, so > >> that statement is a bit misleading. > > > > You are correct, it is misleading, and relies on a few unstated > > assumptions. > > > > In my own personal delving into process splitting, RPC, etc., I usually > > end up with one of two cases; I need really fast call/return, or I need > > not slow call/return. The not slow call/return is (in my opinion) > > satisfactorally solved with XML-RPC. But I've personally not been > > satisfied with the speed of any remote 'fast call/return' packages, as > > they usually rely on cPickle or marshal, which are slow compared to > > even moderately fast 100mbit network connections. When we are talking > > about local connections, I have even seen cases where the > > cPickle/marshal calls can make it so that forking the process is faster > > than encoding the input to a called function. > > This is hard to believe. I've been in that business for a few > years and so far have not found an OS/hardware/network combination > with the mentioned features. > > Usually the worst part in performance breakdown for RPC is network > latency, ie. time to connect, waiting for the packets to come through, > etc. and this parameter doesn't really depend on the OS or hardware > you're running the application on, but is more a factor of which > network hardware, architecture and structure is being used. > > It also depends a lot on what you send as arguments, of course, > but I assume that you're not pickling a gazillion objects :-) > > > I've had an idea for a fast object encoder/decoder (with limited support > > for certain built-in Python objects), but I haven't gotten around to > > actually implementing it as of yet. > > Would be interesting to look at. > > BTW, did you know about http://sourceforge.net/projects/py-xmlrpc/ ? > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Oct 11 2006) > >>> Python/Zope Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > > ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: > > > ------------------------------ >

1 0

Why spawnvp not implemented on Windows?
by Alexey Borzenkov 13 Oct '06

13 Oct '06

Hi all, I've been looking at python 2.5 today and what I notices is absense of spawnvp with this comment in os.py: # At the moment, Windows doesn't implement spawnvp[e], # so it won't have spawnlp[e] either. I'm wondering, why so? Searching MSDN I can see that these functions are implemented in CRT: spawnvp: http://msdn2.microsoft.com/en-us/library/275khfab.aspx spawnvpe: http://msdn2.microsoft.com/en-us/library/h565xwht.aspx I can also see that spawnvp and spawnvpe are currently wrapped in posixmodule.c, but for some reason on OS/2 only. Forgive me if I'm wrong but shouldn't it work when #if defined(PYOS_OS2) is changed to #if defined(PYOS_OS2) || defined(MS_WINDOWS) around spawnvp and spawnvpe wrappers and in posix_methods? At least when I did it with my copy, nt.spawnvp seems to work fine...

9 14

Re: [Python-Dev] Proposal: No more standard library additions
by Giovanni Bajo 13 Oct '06

13 Oct '06

Antoine wrote: >> The standard library is not about easeness of installation. It is >> about having >> a consistent fixed codebase to work with. I don't want to go >> Perl/CPAN, where you have 3-4 alternatives to do thing A which will >> never interoperate >> with whatever you chose among the 3-4 alternatives to do thing B. > > Currently in Python: > http://docs.python.org/lib/module-xml.dom.html > http://docs.python.org/lib/module-xml.dom.minidom.html > http://docs.python.org/lib/module-xml.sax.html > http://docs.python.org/lib/module-xml.parsers.expat.html > http://docs.python.org/lib/module-xml.etree.ElementTree.html > > The problem of "consistent fixed codebase" is that standards get > higher, so eventually those old stable modules lose popularity in > favor of newer, better modules. Those are different paradigms of "doing XML". For instance, the standard library was missing a "pythonic" library to do XML processing, and several arose. ElementTree (fortunately) won and joined the standard distribution. This should allievate the need for other libraries in future. Instead of looking what we have inside, look outside. There are dozens of different XML "pythonic" libraries. I have fought in the past with programs that required large XML frameworks, that in turn required to be downloaded, built, installed, and *understood* to make the required modifictions to the programs themselves. This slowed down my own development, and caused infinite headaches before of version compatibilities (A requires the XML library B, but only versions < 1.2, otherwise you can use A 2.0, which needs Python 2.4+, and then you can use latest B; etc. etc. repeat and complicate ad-libitum). A single version number (that of Python) and a large fixed set of libraries anybody can use is a *strong* PLUS. Then, there is the opposite phenomenom, which is interesting as well. I met many perl programmers which simply re-invented their little wheel everytime. They were mostly system administrators, so they *knew* very well what hell the dependency chains are for both programmers and users. Thus, since perl does not have a standard library, they simply did not import *any* module. This way, the program is "easier" to ship, distribute and use, but it's harder to code, read, fix, and contain unnecessary duplications with everybody's else script. Need to send an e-mail? Why using a library, just paste chunks of cut&pasted mail headers (with MIME, etc.) and do some basic string substitution; and the SMTP protocol is easy, just open a socket a dump some strings to it; or you can use 'sendmail' which is available on any UNIX (and there it goes portability, just because they did not want to evaluate and choose one of the 6 Perl SMTP libraries... and rightfully so!). > Therefore, you have to obsolete old stuff if you want there to be > only One Obvious Way To Do It. I'm totally in favor of obsoletion and removal of old cruft from the standard library. I'm totally against *not* having a standard library. Giovanni Bajo

1 1