Mailman 3 August 2006 - Python-Dev

zip -> izip; is __length_hint__ required?
by Guido van Rossum Aug. 27, 2006

Aug. 27, 2006

At today's sprint, Brian Holmes contributed a patch that implements zip as an interator, a la izip. When reviewing Brian's code, I noticed that he added an implementation of __length_hint__. My gut feeling is that this isn't particularly useful given that zip() is almost exclusively used iteratively, and rarely if ever converted to a list or a tuple. (The one common exception is in the test suite, but there it's almost always a short list, and 3 out of 5 were actually tests for zip or izip.) Should we rip it out or keep it? Also, the existing convention for calling __length_hint__ (e.g. in _PyObject_LengthHint() in abstract.c) seems to be to use PyObject_CallMethod() and suppress TypeError and AttributeError coming out of the call. It would seem to make much more sense to check whether the attribute exists without calling it, and once it exists, just call it and not suppress any exceptions that come out of it. Is there any reason why this shouldn't work? -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser Aug. 27, 2006

Aug. 27, 2006

Patch / Bug Summary ___________________ Patches : 407 open ( +3) / 3393 closed (+17) / 3800 total (+20) Bugs : 888 open (+28) / 6145 closed (+14) / 7033 total (+42) RFE : 232 open ( +3) / 236 closed ( +1) / 468 total ( +4) New / Reopened Patches ______________________ most missing from Windows distribution (2006-08-16) CLOSED http://python.org/sf/1541412 opened by Jim Jewett buffer overrun in repr() for unicode strings (2006-08-16) CLOSED http://python.org/sf/1541585 opened by Simon Law work around clocks with low resolution (uuid.py) (2006-08-17) http://python.org/sf/1541863 opened by Hirokazu Yamamoto patch in bug 1542016 (2006-08-17) CLOSED http://python.org/sf/1542019 opened by Santiago Gala fix crash with continue in nested try/finally (2006-08-18) http://python.org/sf/1542451 opened by Neal Norwitz Improve dynamic linking support on AIX (2006-08-18) http://python.org/sf/1542544 opened by Göran Uddeborg Tweak pydoc to speak about with and CONTEXTMANAGERS (2006-08-18) http://python.org/sf/1542681 opened by Santiago Gala Fix the vc8 solution files (2006-08-18) http://python.org/sf/1542946 opened by baus urllib2 regression (2006-08-19) CLOSED http://python.org/sf/1542948 opened by John J Lee tarfile.py: fix for bug 1543303 (2006-08-21) CLOSED http://python.org/sf/1543897 opened by Lars Gustäbel Socket module is not thread-safe (2006-08-21) http://python.org/sf/1544279 opened by Maxim Sobolev Implementation of PEP 362 (2006-08-22) http://python.org/sf/1544909 opened by Brett Cannon Fixes SocketServer Bug 1531963 (2006-08-23) http://python.org/sf/1545011 opened by Damon Kohler new splicetee module (2006-08-23) http://python.org/sf/1545262 opened by Omar AitMous new directio module (2006-08-23) http://python.org/sf/1545275 opened by Omar AitMous xrange that supports longs, etc (2006-08-24) http://python.org/sf/1546078 opened by Neal Norwitz crash in dict_equal (2006-08-24) http://python.org/sf/1546288 opened by Guido van Rossum Create a real zip iterator object; not using itertools.izip (2006-08-24) CLOSED http://python.org/sf/1546297 opened by Brian Holmes broken error reporting when filename specified by -s missing (2006-08-25) http://python.org/sf/1546372 opened by lplatypus Patches Closed ______________ most missing from Windows distribution (2006-08-16) http://python.org/sf/1541412 closed by jimjjewett buffer overrun in repr() for unicode strings (2006-08-16) http://python.org/sf/1541585 closed by gbrandl broken shortcut keys (2006-08-15) http://python.org/sf/1540874 closed by kbk patch in bug 1542016 (2006-08-17) http://python.org/sf/1542019 closed by gbrandl urllib2 regression (2006-08-19) http://python.org/sf/1542948 closed by gbrandl Backports from trunk to release24-maint (2006-08-15) http://python.org/sf/1540329 closed by gbrandl tarfile.py: fix for bug 1543303 (2006-08-21) http://python.org/sf/1543897 closed by nnorwitz Add some explication to PEP 3100 (2006-07-13) http://python.org/sf/1522038 closed by bcannon Create a real zip iterator object; not using itertools.izip (2006-08-24) http://python.org/sf/1546297 closed by gvanrossum Add readinto method to StringIO and cStringIO (2006-08-12) http://python.org/sf/1539381 closed by bcannon Remove the repr()-backticks (2006-06-04) http://python.org/sf/1500623 closed by bcannon Move reduce() to functools (2006-06-28) http://python.org/sf/1513870 closed by gvanrossum New / Reopened Bugs ___________________ infinite __str__ recursion in thread causes seg fault (2003-07-31) http://python.org/sf/780714 reopened by gbrandl tools and demo missing from windows (2006-08-16) http://python.org/sf/1541420 opened by Jim Jewett bug in python 2.4.3 for windows? (2006-08-16) CLOSED http://python.org/sf/1541566 opened by tinkiwinky Compiler command percent-sign causes format string error (2006-08-16) CLOSED http://python.org/sf/1541642 opened by Matthew Cowles bsddb can't use unicode filenames (2006-08-17) http://python.org/sf/1541671 opened by Zooko O'Whielacronx Incorrect example calls to PyObject_SetItem (2006-08-17) CLOSED http://python.org/sf/1541682 opened by John Machin Recently introduced sgmllib regexp bug hangs Python (2006-08-17) http://python.org/sf/1541697 opened by John J Lee Exceptions don't call _PyObject_GC_UNTRACK(self) (2006-08-17) http://python.org/sf/1542051 opened by iga Seilnacht global variable: multiple id()-addresses (2006-08-17) CLOSED http://python.org/sf/1542166 opened by Frank R. Schaefer Nested finally in generators don't follow PEP 342 (2006-08-17) http://python.org/sf/1542308 opened by Bob Ippolito httplib reads one byte per system call (2006-08-18) http://python.org/sf/1542407 opened by Zoyd Wheeler python-2.5c1.msi contains ICE validation errors and warnings (2006-08-18) http://python.org/sf/1542432 opened by Bob Arnson Bug in definition of PyImport_ImportModuleEx macro (2006-08-18) CLOSED http://python.org/sf/1542693 opened by Chris Stawarz idle in python 2.5c1 freezes on macos 10.3.9 (2006-08-18) http://python.org/sf/1542949 opened by David Strozzi tarfile in mode w|gz adds padding that annoys gunzip (2006-08-19) CLOSED http://python.org/sf/1543303 opened by alexis "from __future__ import foobar;" causes wrong SyntaxError (2006-08-20) http://python.org/sf/1543306 opened by daniel hahler Operator precedence inconsistent for complex literal (2006-08-20) CLOSED http://python.org/sf/1543347 opened by [N/A] test_tempfile fails on cygwin (2006-08-20) http://python.org/sf/1543467 opened by Miki Tebeka test_subprocess fails on cygwin (2006-08-20) http://python.org/sf/1543469 opened by Miki Tebeka md5 sums are different between Solaris and Windows XP SP1 (2006-08-21) http://python.org/sf/1543801 reopened by sunmountain md5 sums are different between Solaris and Windows XP SP1 (2006-08-21) http://python.org/sf/1543801 opened by Stefan Sonnenberg ctypes unit test fails (test_macholib.py) under MacOS 10.4.7 (2006-08-21) http://python.org/sf/1544102 opened by M. J. Fromberger test_anydbm segmentation fault (2006-08-21) http://python.org/sf/1544106 opened by Clay Spence python compiler support for with stmt (2006-08-21) http://python.org/sf/1544277 opened by Neal Norwitz Fix Lib/test/test___all__.py (2006-08-21) http://python.org/sf/1544295 opened by Hasan Diwan checking size of int... configure: error: cannot compute siz (2006-08-21) http://python.org/sf/1544306 opened by arrecostao _ctypes fails to build on Solaris x86 32-bit (Sun compiler) (2006-08-21) http://python.org/sf/1544339 opened by Case Van Horsen Bad result of calculation (2006-08-22) CLOSED http://python.org/sf/1544381 opened by Jean-Christophe BERTOLINI x!=y and [x]=[y] (!) (2006-08-22) http://python.org/sf/1544762 opened by Alex Martelli setup() keyword have to be list (doesn't work with tuple) (2006-08-23) http://python.org/sf/1545341 opened by STINNER Victor New-style classes fail to cleanup attributes (2006-08-23) http://python.org/sf/1545463 opened by Alexander Belopolsky inconsistent treatment of NULs in int() (2006-08-23) http://python.org/sf/1545497 opened by Neal Norwitz distutils home scheme lacks python versioning (2006-08-24) http://python.org/sf/1545658 opened by John Levon distutils needs vendor-packages support (2006-08-24) http://python.org/sf/1545659 opened by John Levon gcc trunk (4.2) exposes a signed integer overflows (2006-08-24) http://python.org/sf/1545668 opened by Jack Howarth structmember T_LONG won't accept a python long (2006-08-24) http://python.org/sf/1545696 opened by Roger Upole Incomplete info in 7.18.1 ZipFile Objects (2006-08-24) http://python.org/sf/1545836 opened by Taco array.array borks on deepcopy (2006-08-24) http://python.org/sf/1545837 opened by Václav Haisman PyString_FromString() clarification (2006-08-24) http://python.org/sf/1546052 opened by Christian Walther bcannon secure branch issues (2006-08-24) CLOSED http://python.org/sf/1546203 opened by Jim Jewett subprocess.Popen can't read file object as stdin after seek (2006-08-25) http://python.org/sf/1546442 opened by GaryD String methods don't support explicit None arguments (2006-08-25) http://python.org/sf/1546585 opened by Nick Coghlan urlparse.urljoin odd behaviour (2006-08-25) http://python.org/sf/1546628 opened by Andres Riancho Bugs Closed ___________ Python 2.5 svn crash in _elementtree.c (2006-08-04) http://python.org/sf/1534630 closed by effbot bug in python 2.4.3 for windows? (2006-08-16) http://python.org/sf/1541566 closed by gbrandl Compiler command percent-sign causes format string error (2006-08-16) http://python.org/sf/1541642 deleted by mdcowles Incorrect example calls to PyObject_SetItem (2006-08-17) http://python.org/sf/1541682 closed by gbrandl global variable: multiple id()-addresses (2006-08-17) http://python.org/sf/1542166 closed by gbrandl Bug in definition of PyImport_ImportModuleEx macro (2006-08-18) http://python.org/sf/1542693 closed by nnorwitz tarfile in mode w|gz adds padding that annoys gunzip (2006-08-19) http://python.org/sf/1543303 closed by nnorwitz Operator precedence inconsistent for complex literal (2006-08-20) http://python.org/sf/1543347 closed by gbrandl md5 sums are different between Solaris and Windows XP SP1 (2006-08-21) http://python.org/sf/1543801 closed by sunmountain Bad result of calculation (2006-08-22) http://python.org/sf/1544381 closed by gbrandl bcannon secure branch issues (2006-08-24) http://python.org/sf/1546203 closed by bcannon New / Reopened RFE __________________ Optimizations for cgi.FieldStorage methods (2006-08-16) http://python.org/sf/1541463 opened by Bob Kline wsgi.org link in wsgiref (2006-08-18) http://python.org/sf/1542920 opened by Ian Bicking RFE Closed __________ extend strptime to understand logging timestamps (2004-08-10) http://python.org/sf/1006786 closed by bcannon

1 0

Re: [Python-Dev] gcc 4.2 exposes signed integer overflows
by howarth＠bromo.msbb.uc.edu Aug. 26, 2006

Aug. 26, 2006

Guido, You'll never win that argument with the gcc developers. If you rely on undefined behavior in the c language standard, they have in the past, and will continue to, feel free to ignore those cases. If you plan on ignoring this issue, just be prepared to see a testcase failure in the python testsuite when python is built with gcc 4.2. Jack

2 1

Removing anachronisms from logging module
by Guido van Rossum Aug. 26, 2006

Aug. 26, 2006

The logging module currently contains some weird-looking code that is there to maintain 1.5.2 compatibility (*). I don't think that's needed any more; neither does Vinay. I propose as a project for someone to clean it up; perhaps compatibility could be maintained with 2.2 but that's probably as far back as it needs to go. Linux distros all come with 2.2 or newer these days. The logging module is currently listed in PEP 291 as requiring compatibility with 1.5.2; my proposal is to remove this or replace it with 2.2 (Vinay even proposed 2.4). (*) I found an example of code testing "if string.find(s, t) != 0", thinking it was a bug attempting to write "if t in s", but which Vinay identified as a 1.5.2 idiom for "if not s.startswith(t)"... -- --Guido van Rossum (home page: http://www.python.org/~guido/)

3 4

Need help with test_mutants.py
by Guido van Rossum Aug. 24, 2006

Aug. 24, 2006

There's a unit test "test_mutants" which I don't understand. If anyone remembers what it's doing, please contact me -- after ripping out dictionary ordering in Py3k, it stops working. In particular, the code in test_one() requires changes, but I don't know how... Please help! -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2 4

for 2.5 issues
by Neal Norwitz Aug. 24, 2006

Aug. 24, 2006

I don't want to make any more changes to 2.5 unless we absolutely have to. I also don't want to lose fixes. How about for anything that should be resolved in 2.5, but wait for 2.5.1 we set the tracker item to: Group 2.5, Resolution: Later, Priority 7. Then it should be easy to find these things. Thanks, n

1 0

Can LOAD_GLOBAL be optimized to a simple array lookup?
by K.S.Sreeram Aug. 24, 2006

Aug. 24, 2006

Hi all, I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup, and was wondering if that can be optimized to a simple array lookup. If i'm right there are 3 kinds of name lookups: locals, outer scopes(closures), and globals. (not counting attribute lookup). Locals are identified by, either the presence of assignments, or their presence in the arg list. So all name lookups can be classified into the 3 types at compile/load time. Since we know, at load time, which names are global.. Can't we simply build a global name table and replace LOAD_GLOBALs with a lookup at the corresponding index into the global name table? The module's dict object will need to be special so that whenever a name gets 'set', the global name table should get updated. Is this optimization possible or have i missed something? cheers [sreeram;]

6 7

xrange accepting non-ints
by Neal Norwitz Aug. 24, 2006

Aug. 24, 2006

I've been working on enhancing xrange and there are a bunch of issues to consider. I've got pretty much complete implementations in both C and Python. Currently xrange is 2 objects: range and the iter. These only work on C longs. Here's what I propose: 2.6: * Add deprecation warning if a float is passed to xrange (currently silently truncates) * Allow int, long, float, or obj.__index__ * Implement xrange in python * Implement iterator over C longs (or Py_ssize_t) in C * Implement iterator over Python longs in Python (* may lose __length_hint__) * Capture the values on construction, so mutating objects wouldn't affect xrange The idea is to expand xrange's capabilities so that it can replace range in 3k. I've profiled various combinations. Here are the various results normalized doing xrange(0, 1e6, 1): Run on all integer (32-bit) values for start, step, end: C xrange and iter: 1 Py xrange w/C iter: 1 Py xrange w/Py iter (gen): 5-8 Py xrange w/Py iter (class): ~30 So having xrange in python is the same speed as if xrange is written in C. The important part is that the iterator needs to be written in C for speed. If we use a generator, something like: while value < end: yield value value += step The result is ~5 times slower in a release build and 8 times slower in a debug build than with an iterator implemented in C. Using a generator means that there is no __length_hint__. If we go with a full class that has a __length_hint__ the result was ~32 times slower in a debug build. The Python impl is about 1/10th the size of the C impl, though is lacking some comments. Run on Python longs the result is somewhat interesting. The Python based iterator is faster. There's probably a bug in the C version, but given that there is a lot of object allocation, I wouldn't expect the C version to ever be much faster than a similar Python version. Plus the Python version is trivial (same as above) for ints or longs. The C version for longs is quite a bit of code. Run on all Python longs (still 0..1e6, but sys.maxint..(sys.maxint + 1e6) is the same): C xrange and iter: 1.4 Py xrange w/C iter: not tested Py xrange w/Py iter (gen): 1 Py xrange w/Py iter (class): 4 Caveats: * The generator version above doesn't support floats. We could easily support floats with a different calculation that would be slightly more expensive, but not have accumulated error. * By using the generator version, __length_hint__ gets dropped. This means that converting the iterator into a sequence could be slightly more costly as we have to increase the allocation. This would only happen if any of start, step, end weren't an int. * With a python implementation there is a little bit of bootstraping that is necessary to get the iter implemented in C into the xrange object implemented in Python * Since xrange is implemented in Python, it can be changed. * The Python code is much easier to understand than the C code (there is at least one bug in the current C version where -sys.maxint -1 isn't always displayed properly). Hopefully this is all understandable. If I left anything out, Thomas will remind me. n

4 3

PEP 362 open issues
by Brett Cannon Aug. 24, 2006

Aug. 24, 2006

I have been spending my Google sprint time on writing and implementing PEP 362 (http://www.python.org/dev/peps/pep-0362/), which finally defines Signature objects for functions. With the implementation at a place I am happy with, I wanted to ask about the open issues with the PEP. The first question is whether to have Signature objects on all functions by default or to leave it in the 'inspect' module and have it be something totally optional. My vote is the latter for now and we can add them to all functions if they turn out to be useful and popular. The second open issue is how to handle binding arguments to tuple parameters for the Signature.bind() method. If you have the function ``def foo((a, b): pass``, how do you have the Signature.bind() method handle 'a', and 'b'? You can either take the proposed arguments and in the dict mapping arguments to parameters have an entry for 'a' and 'b' individually, or have an entry for '(a, (b,))'. My vote is the latter since you cannot address 'a' or 'b' individually as keyword arguments and this makes more sense with function annotations since they won't apply to 'a' and 'b' individually but instead to the entire tuple. Either way there is the issue of exhausting an iterator (like a generator) trying to find out if it is the proper length to unpack, for which I raise a TypeError instead of exhaust a possibly mutable iterator currently. -Brett

3 3

Re: [Python-Dev] GeneratorExit is unintuitive and uneccessary
by Phillip J. Eby Aug. 24, 2006

Aug. 24, 2006

At 01:44 AM 8/24/2006 +0200, Igor Bukanov wrote: >Regarding yield in the finally problem: > >The current implementation throws RuntimeException after the yield >transfer the control back to the close method. If that is changed to >reporting a bad behavior and rethrowing GeneratorExit or its hidden >analog as suggested at the point of yield inside the generator, that >would nicely unwind possible outer finally blocks and close the >generator in any case. This approach has already been proposed and rejected multiple times. A badly-behaving generator is broken and asking it to handle the error of its own brokenness will lead only to grief. Once the generator yields inappropriately, it has proven it cannot be trusted. It is only compounding the generator's insanity to ask it to handle this error!

1 0