Mailman 3 March 2009 - Python-Dev

Forgotten Py3.0 change to remove Queue.empty() and Queue.full()
by Raymond Hettinger March 8, 2009

March 8, 2009

Just notices that the empty() and full() methods were still there. IIRC, we agreed to take them out (but leaving qsize() exposed). The docs entries and test cases were taken out, but the actual methods were accidentally left in. Raymond

10 27

Pickler/Unpickler API clarification
by Collin Winter March 8, 2009

March 8, 2009

I'm working on some performance patches for cPickle, and one of the bigger wins so far has been replacing the Pickler's memo dict with a custom hashtable (and hence removing memo's getters and setters). In looking over this, Jeffrey Yasskin commented that this would break anyone who was accessing the memo attribute. I've found a few examples of code using the memo attribute ([1], [2], [3]), and there are probably more out there, but the memo attribute doesn't look like part of the API to me. It's only documented in http://docs.python.org/library/pickle.html as "you used to need this before Python 2.3, but don't anymore". However: I don't believe you should ever need this attribute. The usages of memo I've seen break down into two camps: clearing the memo, and wanting to explicitly populate the memo with predefined values. Clearing the memo is recommended as part of reusing Pickler objects, but I can't fathom when you would want to reuse a Pickler *without* clearing the memo. Reusing the Pickler without clearing the memo will produce pickles that are, as best I can see, invalid -- at least, pickletools.dis() rejects this, which is the closest thing we have to a validator. Explicitly setting memo values has the same problem: an easy, very brittle way to produce invalid data. So my questions are these: 1) Should Pickler/Unpickler objects automatically clear their memos when dumping/loading? 2) Is memo an intentionally exposed, supported part of the Pickler/Unpickler API, despite the lack of documentation and tests? Thanks, Collin [1] - http://google.com/codesearch/p?hl=en#Qx8E-7HUBTk/trunk/google/appengine/api… [2] - http://google.com/codesearch/p?hl=en#M-DDI-lCOgE/lib/python2.4/site-package… [3] - http://google.com/codesearch/p?hl=en#l_w_cA4dKMY/AtlasAnalysis/2.0.3-LST-1/…

6 18

Re: [Python-Dev] draft 3.1 release schedule
by Mitchell L Model March 7, 2009

March 7, 2009

Would whoever is responsible for IDLE please take a look at the patches I submitted for Python 2 & 3 [tracker IDs 5233 and 5234 respectively]. These change the behavior of IDLE so that IDLESTARTUP or PYTHONSTARTUP files are executed with each restart. This allows loading frequently used packages, personal utilities, etc. automatically at each restart. I consider this a very important problem in IDLE, especially when using it to teach. I would really like to see them in 3.1. The patch is already there; someone just has to do whatever gets done with patches to validate it and check it in. It's not a lot of code changes.

9 13

Re: [Python-Dev] IMAP/POP3 servers on python.org?
by Aahz March 7, 2009

March 7, 2009

[moving discussion to python-dev from pydotorg] On Fri, Mar 06, 2009, Bill Janssen wrote: > Michael Foord <michael(a)voidspace.org.uk> wrote: >> Bill Janssen wrote: >>> Steve Holden <steve(a)holdenweb.com> wrote: >>>> >>>> Seems to me it might be better to have a test start a local server then >>>> kill it, but I am presuming you have some good reason why this is not >>>> practical? >>> >>> It seems a steep learning curve just to run the Python test suite, to >>> have to know how to install an IMAP server on your machine. But perhaps >>> you're right. >> >> Does it need to test against a real server - can't some of the lower >> calls be mocked out? > > Not really. These *are* the lower-level calls. One thing I haven't seen addressed in this discussion is why it's undesirable to ship Twisted as part of the testing source tree. Yes, I realize that it could create an "attractive nuisance", but I think the gains in simplifying testing outweigh that. -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson

2 1

Python wins Linux New Media Award for Best Open Source Programming Language
by Michael Foord March 7, 2009

March 7, 2009

Hello all, Not sure if this is the same as the LinuxQuestions award, but it looks different: (German) http://www.linux-magazin.de/news/cebit_2009_openstreetmap_erntet_zwei_linux… I particularly like this snippet from the google translation: The prize was Martin von Löwis of the Python Foundation on behalf of the Python community itself. http://translate.google.co.uk/translate?hl=en&sl=de&u=http://www.linux-maga… All the best, Michael Foord -- http://www.ironpythoninaction.com/

5 4

Windows 2.6.1 installer - msiexec not creating msvcr90.dll
by charlie March 6, 2009

March 6, 2009

I am trying to script a Python installation on Windows, using msiexec from the windows cmd prompt. I do not want to register extensions. I have tried all the combinations I can find on the following page: http://www.python.org/download/releases/2.5/msi/ But, no matter how I run msiexec, it seems that the msvcr90.dll in not created in the Python26 directory. If I double click the msi installer and run through it manually, the msvcr90.dll is created. Is there a way to run msiexec that results in msvc90.dll (and the manifest file) getting created? Thanks in advance, -cjl

2 2

Summary of Python tracker Issues
by Python tracker March 6, 2009

March 6, 2009

ACTIVITY SUMMARY (02/27/09 - 03/06/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2364 open (+31) / 14890 closed (+17) / 17254 total (+48) Open issues with patches: 822 Average duration of open issues: 661 days. Median duration of open issues: 398 days. Open Issues Breakdown open 2337 (+29) pending 27 ( +2) Issues Created Or Reopened (51) _______________________________ subprocess leaves open fds on construction error 03/03/09 CLOSED http://bugs.python.org/issue5179 reopened ocean-city patch mmap and exception type 02/27/09 http://bugs.python.org/issue5384 created ocean-city mmap can crash after resize failure (windows) 02/27/09 CLOSED http://bugs.python.org/issue5385 created ocean-city patch mmap can crash with write_byte 02/27/09 CLOSED http://bugs.python.org/issue5386 created ocean-city mmap.move crashes by integer overflow 02/27/09 http://bugs.python.org/issue5387 created ocean-city Green-box doc glitch: winhelp version only 02/27/09 http://bugs.python.org/issue5388 created tjreedy Uninitialized variable may be used in PyUnicode_DecodeUTF7Statef 02/27/09 CLOSED http://bugs.python.org/issue5389 created gvanrossum Item 'Python x.x.x' in Add/Remove Programs list still lacks an i 03/05/09 http://bugs.python.org/issue5390 reopened loewis mmap: read_byte/write_byte and object type 02/28/09 http://bugs.python.org/issue5391 created ocean-city patch stack overflow after hitting recursion limit twice 02/28/09 http://bugs.python.org/issue5392 created gagenellina patch cmath.cos and cmath.cosh have "nResult" typo in help 02/28/09 CLOSED http://bugs.python.org/issue5393 created mnewman Distutils in trunk does not work with old Python (2.3 - 2.5) 02/28/09 http://bugs.python.org/issue5394 created akitada patch array.fromfile not checking I/O errors 02/28/09 http://bugs.python.org/issue5395 created aguiar os.read not handling O_DIRECT flag 02/28/09 http://bugs.python.org/issue5396 created aguiar PEP 372: OrderedDict 03/01/09 CLOSED http://bugs.python.org/issue5397 created rhettinger patch strftime("%B") returns a String unusable with unicode 03/01/09 CLOSED http://bugs.python.org/issue5398 created t.steinruecken wer 03/01/09 CLOSED http://bugs.python.org/issue5399 created ajaksu2 patches for multiprocessing module on NetBSD 03/01/09 http://bugs.python.org/issue5400 created aniou patch mimetypes.MAGIC_FUNCTION performance problems 03/01/09 CLOSED http://bugs.python.org/issue5401 created aronacher patch, patch, easy, needs review MutableMapping code smell (see OrderedDict) 03/02/09 CLOSED http://bugs.python.org/issue5402 created jimjjewett test_md5 segfault 03/02/09 CLOSED http://bugs.python.org/issue5403 created ocean-city patch Cross-compiling Python 03/02/09 http://bugs.python.org/issue5404 created kumba There is no way of determining which ABCs a class is registered 03/02/09 http://bugs.python.org/issue5405 created pmoore asyncore doc issue 03/02/09 http://bugs.python.org/issue5406 created giampaolo.rodola Broken Py3.1 release build in Visual Studio 2005 03/02/09 http://bugs.python.org/issue5407 created rhettinger test_osx_env failing 03/02/09 CLOSED http://bugs.python.org/issue5408 created benjamin.peterson ConfigParser get<int/float/boolean> methods broken 03/03/09 CLOSED http://bugs.python.org/issue5409 created Absynthe msvcrt bytes cleanup 03/03/09 http://bugs.python.org/issue5410 created ocean-city patch add xz compression support to distutils 03/03/09 http://bugs.python.org/issue5411 created proyvind extend configparser to support [] syntax 03/03/09 http://bugs.python.org/issue5412 created jkaufman patch urllib ctypes error on Mac OS X Server 10.5 03/03/09 http://bugs.python.org/issue5413 created atiware(a)gmx.net asciibin.a2b_uu returns unexpected values on non ascii data 03/04/09 http://bugs.python.org/issue5414 created dmajnemer uuid module generates incorrect values for uuid3 (and possibly u 03/04/09 http://bugs.python.org/issue5415 created pwr str.replace does strange things when given a negative count 03/04/09 http://bugs.python.org/issue5416 created dmajnemer Reference to missing(?) function in Extending & Embedding Docume 03/04/09 http://bugs.python.org/issue5417 created aydt urllib.response.addinfourl does not support __exit__ 03/05/09 http://bugs.python.org/issue5418 created MLModel urllib.request.open(someURL).read() returns a bytes object so w 03/05/09 http://bugs.python.org/issue5419 created MLModel Queue deprecation warning patch 03/05/09 http://bugs.python.org/issue5420 created tleeuwenburg(a)gmail.com Irritating error message by socket's sendto method 03/05/09 http://bugs.python.org/issue5421 created helduel load pyc file with mbcs file system in update_compiled_module 03/05/09 CLOSED http://bugs.python.org/issue5422 created joexo Exception raised when attempting to call set_charset on an email 03/05/09 http://bugs.python.org/issue5423 created cjw296 Packed IPaddr conversion tests should be extended 03/05/09 http://bugs.python.org/issue5424 created phihag patch 2to3 wrong for types.StringTypes 03/05/09 CLOSED http://bugs.python.org/issue5425 created hagen patch README slight error re OSX 03/05/09 CLOSED http://bugs.python.org/issue5426 created MLModel OSX framework make error: ld: duplicate symbol _PyExc_BlockingIO 03/05/09 CLOSED http://bugs.python.org/issue5427 created MLModel Pyshell history management error 03/05/09 http://bugs.python.org/issue5428 created datamoc Core dumps on the Solaris buildbot 03/05/09 http://bugs.python.org/issue5429 created pitrou patch imaplib: must not replace LF or CR by CRLF in literals 03/06/09 http://bugs.python.org/issue5430 created memeplex cmpfunc in Python 3.0.1 windows installer 03/06/09 CLOSED http://bugs.python.org/issue5431 created Nigel Galloway invalid use of setlocale 02/27/09 CLOSED http://bugs.python.org/issue804543 reopened georg.brandl patch Installed but not listed *.pyo break bdist_rpm 03/01/09 CLOSED http://bugs.python.org/issue1533164 reopened tarek patch Issues Now Closed (52) ______________________ [performance] Too many closed() checkings 478 days http://bugs.python.org/issue1407 pitrou run_setup can fail if the setup script uses __file__ 347 days http://bugs.python.org/issue2385 tarek patch test issue 305 days http://bugs.python.org/issue2771 ajaksu2 various doc typos 240 days http://bugs.python.org/issue3320 jnoller patch possible deadlock in python IO implementation 197 days http://bugs.python.org/issue3618 benjamin.peterson patch, needs review Bottom buttons of IDLE Preferences Pane on Mac not wide enough f 169 days http://bugs.python.org/issue3883 ronaldoussoren patch Failure building 64-bit Python on Leopard 134 days http://bugs.python.org/issue4165 ronaldoussoren BufferedWriter non-blocking overage 120 days http://bugs.python.org/issue4263 benjamin.peterson patch repr of httplib.IncompleteRead is stupid 110 days http://bugs.python.org/issue4308 benjamin.peterson patch Rewrite the IO stack in C 88 days http://bugs.python.org/issue4565 benjamin.peterson .read() and .readline() differ in failing 83 days http://bugs.python.org/issue4579 benjamin.peterson patch, needs review compile() doesn't ignore the source encoding when a string is pa 82 days http://bugs.python.org/issue4626 haypo patch, needs review optimize bytecode for conditional branches 70 days http://bugs.python.org/issue4715 jyasskin patch utf-16 BOM is not skipped after seek(0) 57 days http://bugs.python.org/issue4862 haypo patch io.TextIOWrapper calls buffer.read1() 45 days http://bugs.python.org/issue4996 haypo subprocess leaves open fds on construction error 0 days http://bugs.python.org/issue5179 ocean-city patch OS X IDLE.app and bin/idle: missing/extra menu options 23 days http://bugs.python.org/issue5194 ronaldoussoren OS X installer: "Update Shell Profile" script is not updated 20 days http://bugs.python.org/issue5224 ronaldoussoren OS X installer: Welcome and Readme files are out-of-date 20 days http://bugs.python.org/issue5226 ronaldoussoren StringIO failure when reading a chunk of text without newlines 18 days http://bugs.python.org/issue5264 benjamin.peterson StringIO can duplicate newlines in universal newlines mode 18 days http://bugs.python.org/issue5265 benjamin.peterson StringIO.read(n) does not enforce requested size in newline mode 18 days http://bugs.python.org/issue5266 benjamin.peterson 3.0.1 crashes in unicode path 17 days http://bugs.python.org/issue5273 ocean-city patch document expected/required behavior of 3.x io subsystem with res 14 days http://bugs.python.org/issue5323 benjamin.peterson array.fromfile() fails to insert values when EOFError is raised 13 days http://bugs.python.org/issue5334 ocean-city patch collections.namedtuple generates code causing PyChecker warnings 10 days http://bugs.python.org/issue5336 rhettinger Unicode control characters are not allowed as identifiers 3 days http://bugs.python.org/issue5358 loewis RO (shorthand for READONLY) gone, not in documentation 3 days http://bugs.python.org/issue5360 georg.brandl adding --quiet to bdist_rpm 1 days http://bugs.python.org/issue5378 tarek Allow Python keywords as keyword arguments for functions. 0 days http://bugs.python.org/issue5382 mrabarnett mmap can crash after resize failure (windows) 6 days http://bugs.python.org/issue5385 ocean-city patch mmap can crash with write_byte 1 days http://bugs.python.org/issue5386 ocean-city Uninitialized variable may be used in PyUnicode_DecodeUTF7Statef 6 days http://bugs.python.org/issue5389 gvanrossum cmath.cos and cmath.cosh have "nResult" typo in help 0 days http://bugs.python.org/issue5393 marketdickinson PEP 372: OrderedDict 2 days http://bugs.python.org/issue5397 forest patch strftime("%B") returns a String unusable with unicode 0 days http://bugs.python.org/issue5398 amaury.forgeotdarc wer 0 days http://bugs.python.org/issue5399 ajaksu2 mimetypes.MAGIC_FUNCTION performance problems 0 days http://bugs.python.org/issue5401 benjamin.peterson patch, patch, easy, needs review MutableMapping code smell (see OrderedDict) 0 days http://bugs.python.org/issue5402 rhettinger test_md5 segfault 1 days http://bugs.python.org/issue5403 ocean-city patch test_osx_env failing 2 days http://bugs.python.org/issue5408 benjamin.peterson ConfigParser get<int/float/boolean> methods broken 0 days http://bugs.python.org/issue5409 draghuram load pyc file with mbcs file system in update_compiled_module 0 days http://bugs.python.org/issue5422 ocean-city 2to3 wrong for types.StringTypes 1 days http://bugs.python.org/issue5425 hagen patch README slight error re OSX 0 days http://bugs.python.org/issue5426 MLModel OSX framework make error: ld: duplicate symbol _PyExc_BlockingIO 0 days http://bugs.python.org/issue5427 benjamin.peterson cmpfunc in Python 3.0.1 windows installer 0 days http://bugs.python.org/issue5431 amaury.forgeotdarc just testing 3179 days http://bugs.python.org/issue400608 ajaksu2 patch invalid use of setlocale 0 days http://bugs.python.org/issue804543 georg.brandl patch Add ResolveFinderAliases to macostools module 1869 days http://bugs.python.org/issue881824 ronaldoussoren Installed but not listed *.pyo break bdist_rpm 1 days http://bugs.python.org/issue1533164 loewis patch mmap.mmap can overrun buffer 630 days http://bugs.python.org/issue1733986 ocean-city patch Top Issues Most Discussed (10) ______________________________ 26 PEP 372: OrderedDict 2 days closed http://bugs.python.org/issue5397 15 Item 'Python x.x.x' in Add/Remove Programs list still lacks an 1 days open http://bugs.python.org/issue5390 14 Installed but not listed *.pyo break bdist_rpm 1 days closed http://bugs.python.org/issue1533164 10 json needs object_pairs_hook 7 days open http://bugs.python.org/issue5381 9 merge json library with latest simplejson 2.0.x 141 days open http://bugs.python.org/issue4136 9 asyncore delayed calls feature 445 days open http://bugs.python.org/issue1641 8 Queue deprecation warning patch 2 days open http://bugs.python.org/issue5420 8 test_osx_env failing 2 days closed http://bugs.python.org/issue5408 8 Broken Py3.1 release build in Visual Studio 2005 4 days pending http://bugs.python.org/issue5407 8 Cross-compiling Python 4 days open http://bugs.python.org/issue5404

1 0

Re: [Python-Dev] [Tracker-discuss] Adding a "Rietveld this" button?
by Daniel (ajax) Diniz March 6, 2009

March 6, 2009

CC'ing python-dev, as more RFEs might be uncovered :) Daniel (ajax) Diniz wrote: > Martin v. Löwis wrote: >> I think a patch (or full file) would be good enough. We could put it >> into the tracker itself, and publish it prominently where people >> upload files. > > I'll post it as a patch and a full file at issue 247 when I get to it. Posted as full file to http://psf.upfronthosting.co.za/roundup/meta/issue247 , Guido suggests the wrapper way so I won't bother with creating patches now. > [snip code] >> Nice! I didn't think of something that complicated (or, rather, >> complicated in a different way): >> >> upload.py --roundup 5428 > > Just to be clear, this will work as if the user passed the following options: > > upload.py --message "[issue5428] Pyshell history management error" > --cc report(a)bugs.python.org > > Right? :) If that is right, it works :) >> That could either fill in a description given by -d, or fetch >> the description from roundup. > > Fetching a description is as easy as fetching a title[1], but I can't > think of a fixed place to look for one (last message? last patch > description?). Maybe we can add another option that tells the script > where to fetch description/content from? Something like " > [-F|--fetch_description] msg83029" ? Works too, for files or messages :) Examples: http://bugs.python.org/issue400608 http://bugs.python.org/issue2771 http://codereview.appspot.com/25073 http://codereview.appspot.com/24075 Thanks again, Daniel

1 0

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa
by Hrvoje Niksic March 6, 2009

March 6, 2009

Greg Ewing wrote: > Antoine Pitrou wrote: > >> For starters, since py3k is supposed to support non-blocking IO, why not write a >> portable API to make a raw file or socket IO object non-blocking? > > I think we need to be clearer what we mean when we talk > about non-blocking in this context. Normally when you're > using select/poll you *don't* make the underlying file > descriptor non-blocking in the OS sense. The non-blockingness > comes from the fact that you're using select/poll to make > sure the fd is ready before you access it. > > So I don't think it makes sense to talk about having a > non-blocking API as a separate thing from a select/poll > wrapper. The select/poll wrapper *is* the non-blocking > API. This is not necessarily the case. In fact, modern sources often recommend using poll along with the non-blocking sockets for (among other things) performance reasons. For example, when a non-blocking socket becomes readable, you don't read from it only once and go back to the event loop, you read from it in a loop until you get EAGAIN. This allows for processing of fast-incoming data with fewer system calls. Linux's select(2) man page includes a similar advice with different motivation: Under Linux, select() may report a socket file descriptor as "ready for reading", while nevertheless a subsequent read blocks. This could for example happen when data has arrived but upon examination has wrong checksum and is discarded. There may be other circumstances in which a file descriptor is spuriously reported as ready. Thus it may be safer to use O_NONBLOCK on sockets that should not block. Even if you don't agree that using O_NONBLOCK with select/poll is the best approach to non-blocking, I think there is enough existing practice of doing this to warrant separate consideration of non-blocking sockets (in the OS sense) and select/poll.

3 2

Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa
by Hrvoje Niksic March 6, 2009

March 6, 2009

Greg Ewing wrote: >> Even if you don't agree that using O_NONBLOCK with select/poll is the >> best approach to non-blocking, I think there is enough existing practice >> of doing this to warrant separate consideration of non-blocking sockets >> (in the OS sense) and select/poll. > > I'm not saying there isn't merit in having support for > non-blocking file descriptors, only that it's not in > any sense a prerequisite or first step towards a > select/poll wrapper. They're orthogonal issues, even > if you might sometimes want to use them together. In that case we are in agreement. Looking back, I was somewhat confused by this paragraph: So I don't think it makes sense to talk about having a non-blocking API as a separate thing from a select/poll wrapper. The select/poll wrapper *is* the non-blocking API. If they're orthogonal, then it does make sense to talk about having a separate non-blocking socket API and poll API, even if the latter can be used to implement non-blocking *functionality* (hypothetical Linux issues aside).

1 0