Mailman 3 August 2008 - Python-Dev

Where are universal newlines handled in the parser/compiler?
by Brett Cannon 17 Aug '08

17 Aug '08

If you import a module that uses '\r\n' line endings, Python does the right thing. But if you read in the bytes for the same file and then pass it to compile() you get an unhelpful syntax error pointing at a blank line. Normally I would say one should just open the source file as 'r' instead of 'rb', but with source code that does not work well as their can be a source encoding set. Lib/test/test_pep263.py is the perfect example of this; Windows newlines with a koi8-r encoding. What I would like to do is get compile() to work properly with a bytes stream just as if Python itself was handling the compilation through import and from a file directly. But before I try to dig into the parser to figure out where the translation of newlines occurs (or where the translation option is set), I thought I would ask to see if anyone just happened to know (I have already spent a few hours figuring out why Latin-1 encodings were not working with compile() so I don't want to go diving into the maze of function calls in the parser again). -Brett

2 1

eval() accepts compiled 'exec' statements
by Patrick Maupin 17 Aug '08

17 Aug '08

This may or may not be relevant to 2.6 or 3.0 (because I don't have those handy at the moment), but on 2.5 and earlier: Python 2.5.2 (r252:60911, May 7 2008, 15:19:09) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print eval(compile('print "It works"\n', '', 'exec')) It works None >>> Personally, I'm absolutely fine with this (because I have a use case, naturally), but eval() does work differently for compiled vs non-compiled objects, and this behavior doesn't match what the doc at http://docs.python.org/lib/built-in-funcs.html says, namely "This function can also be used to execute arbitrary code objects (such as those created by compile()). In this case pass a code object instead of a string. The code object must have been compiled passing 'eval' as the kind argument." That doc statement is, itself, contradictory. How can it be an "arbitrary code object" if it must have been compiled using 'eval'? Perhaps all that is meant is "In order to return a result other than None to eval's caller, the code object must have been compiled passing 'eval' as the kind argument." So, either the code or the doc should be fixed. I would vote +1 for fixing the doc to match the code, but then I'm the sort of dysfunctional programmer who sometimes has a use-case for expressions with side-effects. Thanks, Pat

2 1

Memory tests in Unicode
by Facundo Batista 17 Aug '08

17 Aug '08

The test_raiseMemError() in test_unicode.py is complicating me the regression tests: tries to use all the memory in my system, which starts to swap as crazy, and almost freezes everything. When the test finishes (always pass ok), I have all the memory flushed so it take a few seconds to go back to the normal system. I told myself that this test should run only in some special cases, but didn't find any flag in regrtest.py (like "largefile", or "network"), to put this test under. Are you having also this issue? Do you think that it should run not *always*? It's ok if I add a flag like "memory" that will hold "Tests that use large amount of memory. Normally these tests take long to run because use all your memory and make the system swap"? Thank you!! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

7 9

Implementation of Camellia block cipher for Python Cryptography Toolkit.
by Yoshisato YANAGISAWA 16 Aug '08

16 Aug '08

Hi, I am afraid that here is the right place to say but I implemented a patch to add support for Camellia block cipher to Python Cryptography Toolkit (aka. pycrypto). Camellia is one of the approved encryption methods by European Union (one of the finalist of NESSIE) and has specified in several RFCs. It is also included in some open source softwares such as Linux, OpenSSL, Firefox 3, and so on. I put the patch at: http://www.csg.is.titech.ac.jp/~yanagisawa/Sites/text/camellia/pycrypto-2.0… in http://www.csg.is.titech.ac.jp/~yanagisawa/Sites/text/camellia-e.html Will you please review and test it? I hope pycrypto will include the patch. Thank you in advance, -- Yoshisato Yanagisawa <yanagisawa(a)csg.is.titech.ac.jp>

2 2

Summary of Python tracker Issues
by Python tracker 15 Aug '08

15 Aug '08

ACTIVITY SUMMARY (08/08/08 - 08/15/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1949 open (+18) / 13434 closed (+14) / 15383 total (+32) Open issues with patches: 608 Average duration of open issues: 726 days. Median duration of open issues: 1646 days. Open Issues Breakdown open 1936 (+17) pending 13 ( +1) Issues Created Or Reopened (32) _______________________________ Remove long integer literals from Python 3.0 tutorial 08/09/08 CLOSED http://bugs.python.org/issue3529 created jsizelove patch ast.NodeTransformer bug 08/09/08 http://bugs.python.org/issue3530 created daishiharada file read preallocs 'size' bytes which can cause memory problems 08/09/08 http://bugs.python.org/issue3531 created dalke bytes.tohex method 08/09/08 http://bugs.python.org/issue3532 created mgiuca patch mac 10.4 buld of 3.0 --with-pydebug fails no __eprintf 08/09/08 CLOSED http://bugs.python.org/issue3533 created barry-scott refactor.py can lose indentation for relative imports 08/09/08 CLOSED http://bugs.python.org/issue3534 created rupole zipfile has problem reading zip files over 2GB 08/10/08 http://bugs.python.org/issue3535 created alonwas case conversion problems in Turkish 08/10/08 http://bugs.python.org/issue3536 created ozan dict creation failure causes crash 08/10/08 CLOSED http://bugs.python.org/issue3537 created rupole patch Docstring typos 08/10/08 CLOSED http://bugs.python.org/issue3538 created Rambo007 Problem with pgen make dependencies in certain circumstances 08/11/08 http://bugs.python.org/issue3539 created skip.montanaro NotEmptyErrorError 08/11/08 CLOSED http://bugs.python.org/issue3540 created mft bsddb memory leak on ubuntu 08/11/08 http://bugs.python.org/issue3541 created kcwu Building an MSI installer crashes 08/11/08 http://bugs.python.org/issue3542 created pitrou patch ctypes works on Intel OS X, fails on PPC OS X 08/11/08 CLOSED http://bugs.python.org/issue3543 created janssen "expection" typo 08/11/08 CLOSED http://bugs.python.org/issue3544 created endolith Python turning off assertions (Windows) 08/12/08 CLOSED http://bugs.python.org/issue3545 created abe Missing linebreak in ext.doctest output 08/12/08 CLOSED http://bugs.python.org/issue3546 created schuppenies patch Ctypes is confused by bitfields of varying integer types 08/12/08 http://bugs.python.org/issue3547 created tim.maxwell subprocess.pipe function 08/12/08 http://bugs.python.org/issue3548 created tebeka patch Missing IDLE Preferences on Mac 08/13/08 http://bugs.python.org/issue3549 created pchew Socket Python 3k Documentation mistake OR Unicode string is not 08/14/08 CLOSED http://bugs.python.org/issue3550 created st33med multiprocessing.Pipe terminates with ERROR_NO_SYSTEM_RESOURCES i 08/14/08 http://bugs.python.org/issue3551 created ocean-city uuid - exception on uuid3/uuid5 08/14/08 CLOSED http://bugs.python.org/issue3552 created mgiuca patch "2to3 -l" doesn't work when installed in /opt 08/14/08 CLOSED http://bugs.python.org/issue3553 created haypo patch ctypes.wstring_at and string_at call Python API without the GIL 08/14/08 http://bugs.python.org/issue3554 created kevinwatters Regression: nested exceptions crash (Cannot recover from stack o 08/14/08 http://bugs.python.org/issue3555 created ajaksu2 test_raiseMemError consumes an insane amount of memory 08/14/08 http://bugs.python.org/issue3556 created loewis Segfault in sha1 08/15/08 CLOSED http://bugs.python.org/issue3557 created mgiuca patch Operator precedence misdocumented 08/15/08 http://bugs.python.org/issue3558 created tjreedy Pasted \n not same as typed \n 08/15/08 http://bugs.python.org/issue3559 created tjreedy redundant "base" field in memoryview objects 08/15/08 http://bugs.python.org/issue3560 created pitrou Issues Now Closed (28) ______________________ Can't pickle partial functions 282 days http://bugs.python.org/issue1398 ngrilly Strange behavior of urlparse.urljoin 276 days http://bugs.python.org/issue1432 facundobatista patch trunk version does not compile with vs8 and vc6 185 days http://bugs.python.org/issue2065 loewis patch Windows: "Runtime Error!" crash from pythonw.exe (3.0a5) 90 days http://bugs.python.org/issue2841 tjreedy sphinx: config option for exclude_dirnames 59 days http://bugs.python.org/issue3097 georg.brandl shutil references undefined WindowsError symbol 54 days http://bugs.python.org/issue3134 pitrou patch bytearrays are not thread safe 56 days http://bugs.python.org/issue3139 loewis patch bz2 iterator fails silently on MemoryError 44 days http://bugs.python.org/issue3205 pitrou patch shutil.move bahave unexpected in fat32 39 days http://bugs.python.org/issue3253 pitrou csv.DictReader inconsistency 9 days http://bugs.python.org/issue3436 skip.montanaro patch BufferedWriter not thread-safe 15 days http://bugs.python.org/issue3476 pitrou patch pickle segfault with infinite loop in __getattr__ 8 days http://bugs.python.org/issue3514 alexandre.vassalotti Remove long integer literals from Python 3.0 tutorial 1 days http://bugs.python.org/issue3529 georg.brandl patch mac 10.4 buld of 3.0 --with-pydebug fails no __eprintf 1 days http://bugs.python.org/issue3533 barry-scott refactor.py can lose indentation for relative imports 0 days http://bugs.python.org/issue3534 benjamin.peterson dict creation failure causes crash 1 days http://bugs.python.org/issue3537 georg.brandl patch Docstring typos 0 days http://bugs.python.org/issue3538 georg.brandl NotEmptyErrorError 0 days http://bugs.python.org/issue3540 georg.brandl ctypes works on Intel OS X, fails on PPC OS X 0 days http://bugs.python.org/issue3543 janssen "expection" typo 0 days http://bugs.python.org/issue3544 georg.brandl Python turning off assertions (Windows) 2 days http://bugs.python.org/issue3545 loewis Missing linebreak in ext.doctest output 2 days http://bugs.python.org/issue3546 georg.brandl patch Socket Python 3k Documentation mistake OR Unicode string is not 0 days http://bugs.python.org/issue3550 georg.brandl uuid - exception on uuid3/uuid5 1 days http://bugs.python.org/issue3552 loewis patch "2to3 -l" doesn't work when installed in /opt 1 days http://bugs.python.org/issue3553 benjamin.peterson patch Segfault in sha1 0 days http://bugs.python.org/issue3557 loewis patch Python code.interact() and UTF-8 locale 1063 days http://bugs.python.org/issue1288615 haypo patch Tkinter.Menu.delete doesn't delete command of entry 1015 days http://bugs.python.org/issue1342811 schuppenies patch Top Issues Most Discussed (10) ______________________________ 35 urllib.quote and unquote - Unicode issues 40 days open http://bugs.python.org/issue3300 15 bytearrays are not thread safe 56 days closed http://bugs.python.org/issue3139 13 bytes.tohex method 6 days open http://bugs.python.org/issue3532 11 locale.getpreferredencoding() gives bus error on Mac OS X 10.4. 31 days open http://bugs.python.org/issue3362 9 Array pickling exposes internal memory representation of elemen 150 days open http://bugs.python.org/issue2389 7 Building a Win32 binary installer crashes 55 days open http://bugs.python.org/issue3160 6 mac 10.4 buld of 3.0 --with-pydebug fails no __eprintf 1 days closed http://bugs.python.org/issue3533 6 file read preallocs 'size' bytes which can cause memory problem 7 days open http://bugs.python.org/issue3531 5 Python turning off assertions (Windows) 2 days closed http://bugs.python.org/issue3545 5 zipfile has problem reading zip files over 2GB 5 days open http://bugs.python.org/issue3535

1 0

Re: [Python-Dev] [python-committers] next beta
by M.-A. Lemburg 15 Aug '08

15 Aug '08

First, I'd like to know why this discussion is happening on the committers list. Python-dev is the right list for these things. I've adjusted the CC accordingly. On 2008-08-12 20:44, Martin v. Löwis wrote: >>>> I am planning to offer a single file patch for 2.3 and 2.4. As far as >>>> one more 2.5 release, I don't think there's going to be many changes >>>> to the 2.5 branch between now and 2.6/3.0 final - although if there >>>> is, we'll obviously have to do another release. >>> I would like to establish a tradition where, after some final bug fix >>> release (e.g. for 2.5), further "mere" bug fixes are banned from the >>> maintenance branch (and I did revert several bug fixes from the 2.4 >>> branch). >> I'm not sure I agree with this policy. Can you elaborate on /why/ you >> want this? > > Because there won't typically be sufficient testing and release > infrastructure to allow arbitrary bug fixes to be committed on the > branch. The buildbots are turned off, and nobody tests the release > candidate, no Windows binaries are provided - thus, chances are very > high that a bug fix release for some very old branch will be *worse* > than the previous release, rather than better. Second, I don't think this is true. People using those patch level releases will test and report bugs if they are introduced by such backports. Besides, developers backporting such changes are diligent enough to test their changes - they will usually have a reason for applying the extra effort to backport. I don't see any advantage in undoing already tested and committed patches to an older branch. Note that Python 2.4 is still widely used out there. As an example, all the Zope and Plone installations run Python 2.4 and will continue to do so for quite a while. And there's a reason for this slow uptake of Python 2.5: as more and more servers run 64-bit OSes, the Py_ssize_t changes cause serious trouble with Python C extensions that were not updated by their authors. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 13 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611

8 18

Re: [Python-Dev] [python-committers] PQM?
by Nick Coghlan 15 Aug '08

15 Aug '08

Christian Heimes wrote: > Perhaps we could adopt a release plan similar to Ubuntu. They have > releases with cool, new and bleeding edge stuff followed by a release > that focuses on stability and long term support. Python 2.6 and > especially 3.0 are releases with new features. What do you think about > focusing on stability and long time support for 2.7 and 3.1? 2.7 might > be the last version of the 2.x series and we sure gonna have to fix lots > of issues in the 3.x series until it's matured. I actually thought that was the plan for 2.7/3.1 ever since Guido started talking about Py3k as a real thing instead of a "at some unspecified time in the future" activity. (Well, for 3.1 anyway - 2.6 should be as rock solid as we can make it, but 3.0 is going to have issues with availability of third party extensions no matter how solid or otherwise the core might be) Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org

1 0

Looking for an up-to-date address for john.benediktsson
by Brett Cannon 15 Aug '08

15 Aug '08

I was going through our list of email addreses for committers and realized that I don't have an address for john.benediktsson that works; his ewtllc.com address is bouncing. Anyone have an up-to-date one? -Brett

2 1

Fwd: PEP: Frequently-requested additional features for the `unittest` module
by Olemis Lang 14 Aug '08

14 Aug '08

> > Abstract > ======== > > This PEP proposes frequently-requested additions to the standard > library `unittest` module that are natural extensions of its existing > functionality. > I had not seen this proposal... I will post a few comments. > > Motivation > ========== > > The `unittest` module is functionally complete. Nevertheless, many > users request and devise extensions to perform functions that are so > common they deserve to be in the standard module. > > > Specification > ============= > > New condition tests > ------------------- > > The following test methods will be added to the ``TestCase`` class. > The function signature is part of this specification. The body is to > be treated as a reference implementation only; any functionally > identical implementation also meets this specification. > ... > > The following test methods are also added. Their implementation in > each case is simply the logical inverse of a corresponding method > above. > My question here is... should assert* (fail*) methods be included in a separate class or module? I think this is a good idea (at least in the long term) Take a look at the following thread [1] for further details > > Simple invocation of test collection > ------------------------------------ > > The following new functionality will be added to the `unittest` > module. The function signature for ``run_tests`` is part of this > specification; the implementation is to be considered a reference > implementation only. > > :: > > def run_tests( > tests, > loader_class=TestLoader, suite_class=TestSuite, > runner_class=TextTestRunner): > """ Run a collection of tests with a test runner > > :param tests: > A sequence of objects that can contain test cases: > modules, `TestSuite` instances, or `TestCase` > subclasses. > :param loader_class: > The type of test loader to use for collecting tests > from the `tests` collection. > :param suite_class: > The type of test suite to use for accumulating the > collected test cases. > :param runner_class: > The type of test runner to instantiate for running the > collected test cases. > :return: > None. > > """ > I suppose that the reason for this function to exist is to ease test code (provide a shortcut or something), reuse some test code, and still allow different loading, running strategies. However, I dont think that passing (loader / suite / runner) classes in to this function is a good idea... IMO objects should be considered instead... Why? Mainly because the code assumes that all loaders are stateless. This may be true for unittest's and perhaps others, but this might not always be true. An example of statefull test loaders might be found in [2] (presented in [3]), and perhaps there other examples like this (even though I don't know them). This kind of code might restrain the use of this kind of loaders, and therefore might not be as reusable as should be. Nonetheless, you might think this is a very peculiar case and decide that this comment is irrelevant, but as far as I know, semantics of unittest TestLoader do not enforce that its subclasses ought to be stateless (although, I repeat, unittest test loaders are stateless). The same might be said for test runners... and they are more likely to be stateful. > def iter_testcases_recursively(collection, loader): > # Flatten and iterate over collection, generating > # instances of TestCase > loader = loader_class() > suite = suite_class() > for testcase in iter_testcases_recursively(tests, loader): > suite.add_tests(testcase) > runner = runner_class() > runner.run(suite) Besides... why is it necessary to flatten a hierarchy of test suites containing test cases? Why do you need to put them into another suite if the test loader already does this for us? Isn't it an extra overhead? Also there is a class attribute (`suiteClass`) in TestLoader which allows doing something like this. [1] [Python-Dev] unittest Suggestions (http://mail.python.org/pipermail/python-dev/2008-August/081763.html) [2] Module dutest (https://sourceforge.net/project/showfiles.php?group_id=220287&abmode=1) [3] [Python-Dev] An OO API for doctest / unittest integration... (http://mail.python.org/pipermail/python-dev/2008-August/081761.html) -- Regards, Olemis.

1 0

uuid test fails
by Matt Giuca 14 Aug '08

14 Aug '08

Hi, I thought I'd bring this up on both the tracker and mailing list, since it's important. It seems the test suite breaks as of r65661. I've posted details to the bug tracker and a patch which fixes the module in question (uuid.py). http://bugs.python.org/issue3552 Cheers Matt

1 0