Mailman 3 July 2013 - Python-Dev

Re: [Python-Dev] [Python-checkins] cpython (merge default -> 3.3): merge 3.3 (#18470)
by Victor Stinner 16 Jul '13

16 Jul '13

2013/7/16 benjamin.peterson <python-checkins(a)python.org>: > http://hg.python.org/cpython/rev/8889c9b5dd3a > changeset: 84654:8889c9b5dd3a > branch: 3.3 > parent: 84653:c3a510b22218 > parent: 84652:8a078bf3cf14 > user: Benjamin Peterson <benjamin(a)python.org> > date: Mon Jul 15 19:15:49 2013 -0700 > summary: > merge 3.3 (#18470) This commit merges branches default and 3.3 into 3.3. Usually, we merge branches 3.3 and default into default. Because of this changeset, when I try to merge 3.3 into default, I get a long list of modifications... Victor

1 0

PEP 447: hooking into super() attribute resolution
by Ronald Oussoren 15 Jul '13

15 Jul '13

Hi, I've posted a new update of my proposal to add a way to override the attribute resolution proces by super(). I've rewritten the PEP and implementation based on feedback by Steve Dower. In the current edition of the proposal the hook is a method on the type, defined in a metaclass for types. A simple demo: class meta (type): def __locallookup__(cls, name): return type.__locallookup__(cls, name.upper()) class S (metaclass=meta): def m(self): return 42 def M(self): return "fortytwo" class SS (S): def m(self): return 21 def M(self): return "twentyone" o = SS() print("self", o.m()) print("super", super(SS, o).m()) The last line currently prints "super 42" and would print "super fortytwo" with my proposal. A major open issue: the __locallookup__ method could also be used for normal attribute resolution, but that probably causes issues with attribute caching (see the PEP for details). I haven't worked out yet if it is worthwhile to tweak the proposal to fix the caching issues (even though the proposal currently says that PyObject_GenericGetAttr will use the new method). Fixing the caching issue definitly would help in my primary use case by reducing code duplication, but might end up making the API unnecessarily complex. Ronald PEP: 447 Title: Hooking into super attribute resolution Version: $Revision$ Last-Modified: $Date$ Author: Ronald Oussoren <ronaldoussoren(a)mac.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 12-Jun-2013 Post-History: 2-Jul-2013, ? Abstract ======== In current python releases the attribute resolution of the `super class`_ peeks in the ``__dict__`` attribute of classes on the MRO to look for attributes. This PEP introduces a hook that classes can use to override that behavior for specific classes. Rationale ========= Peeking in the class ``__dict__`` works for regular classes, but can cause problems when a class dynamicly looks up attributes in a ``__getattribute__`` method. This new hook makes it possible to affect attribute lookup for both normal attribute lookup and lookup through the `super class`_. The superclass attribute lookup hook ==================================== Both ``super.__getattribute__`` and ``object.__getattribute__`` (or `PyObject_GenericGetAttr`_ in C code) walk an object's MRO and peek in the class' ``__dict__`` to look up attributes. A way to affect this lookup is using a method on the meta class for the type, that by default looks up the name in the class ``__dict__``. In Python code -------------- A meta type can define a method ``__locallookup__`` that is called during attribute resolution by both ``super.__getattribute__`` and ``object.__getattribute``:: class MetaType (type): def __locallookup__(cls, name): try: return cls.__dict__[name] except KeyError: raise AttributeError(name) from None The example method above is pseudo code for the implementation of this method on `type`_ (the actual implementation is in C, and doesn't suffer from the recursion problem in this example). The ``__locallookup__`` method has as its arguments a class and the name of the attribute that is looked up. It should return the value of the attribute without invoking descriptors, or raise `AttributeError`_ when the name cannot be found. In C code --------- A new slot ``tp_locallookup`` is added to the ``PyTypeObject`` struct, this slot corresponds to the ``__locallookup__`` method on `type`_. The slot has the following prototype:: PyObject* (*locallookupfunc)(PyTypeObject* cls, PyObject* name); This method should lookup *name* in the namespace of *cls*, without looking at superclasses, and should not invoke descriptors. The method returns ``NULL`` without setting an exception when the *name* cannot be found, and returns a new reference otherwise (not a borrowed reference). Usage of this hook ------------------ The new method will be defined for `type`_, and will peek in the class dictionary:: static PyObject* type_locallookup(PyTypeObject* cls, PyObject* name) { PyObject* res; if (!cls->tp_dict) { return NULL; } res = PyDict_GetItem(cls, name); Py_XINCREF(res); return res; } The new method will be used by both `PyObject_GenericGetAttr`_ and ``super.__getattribute__`` instead of peeking in a type's ``tp_dict``. Alternative proposals --------------------- ``__getattribute_super__`` .......................... An earlier version of this PEP used the following static method on classes:: def __getattribute_super__(cls, name, object, owner): pass This method performed name lookup as well as invoking descriptors and was necessarily limited to working only with ``super.__getattribute__``. Reuse ``tp_getattro`` ..................... It would be nice to avoid adding a new slot, thus keeping the API simpler and easier to understand. A comment on `Issue 18181`_ asked about reusing the ``tp_getattro`` slot, that is super could call the ``tp_getattro`` slot of all methods along the MRO. That won't work because ``tp_getattro`` will look in the instance ``__dict__`` before it tries to resolve attributes using classes in the MRO. This would mean that using ``tp_getattro`` instead of peeking the class dictionaries changes the semantics of the `super class`_. Open Issues =========== * The names of the new slot and magic method are far from settled. * Should the python method raise an exception or return a magic value (such as the `NotImplemented`_ return value used by comparison operators). The latter could be slightly faster because it doesn't have to overhead of setting up exception state, but makes it impossible to use that value as an attribute on a class. * The proposed change to `PyObject_GenericGetAttr`_ will probably cause problems with the attribute lookup cache (MCACHE): 1. That code stores borrowed references, which won't work when the hook is present. That is mostly fixable, but at the cost of possibly keeping garbage alive. 2. Caching isn't an option when a hook might execute arbitrary code (and there hence is no reason to assume that the hooks return value won't change later on). The only workaround I could find for this is to make the hook a fallback (that is, more like ``__getattr__`` than ``__getattribute__``). References ========== * `Issue 18181`_ contains a prototype implementation Copyright ========= This document has been placed in the public domain. .. _`Issue 18181`: http://bugs.python.org/issue18181 .. _`super class`: http://docs.python.org/3/library/functions.html#super .. _`NotImplemented`: http://docs.python.org/3/library/constants.html#NotImplemented .. _`PyObject_GenericGetAttr`: http://docs.python.org/3/c-api/object.html#PyObject_GenericGetAttr .. _`type`: http://docs.python.org/3/library/functions.html#type .. _`AttributeError`: http://docs.python.org/3/library/exceptions.html#AttributeError

2 3

Hooking into super() attribute resolution
by Ronald Oussoren 15 Jul '13

15 Jul '13

Hi, Below is a very preliminary draft PEP for adding a special method that can be used to hook into the attribute resolution process of the super object. The primary usecase for using this special method are classes that perform custom logic in their __getattribute__ method, where the default behavior of super (peekin the the class __dict__) is not appropriate. The primary reason I wrote this proposal is PyObjC: it dynamicly looks up methods in its __getattribute__ and caches the result in the class __dict__, because of this super() will often not work correctly and therefore I'm currently shipping a custom subclass of super() that basicly contains an in-line implementation of the hook that would be used by PyObjC. I have a partial implementation of the hook system in issue 18181 and a PyObjC patch that uses it. The implementation currently does not contain tests, and I'm sure that I'll find edge cases that I haven't thought about yet when I add tests. Ronald PEP: TODO Title: Hooking into super attribute resolution Version: $Revision$ Last-Modified: $Date$ Author: Ronald Oussoren <ronaldoussoren(a)mac.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 12-Jun-2013 Post-History: 2-Jul-2013 Abstract ======== In current python releases the attribute resolution of the `super class`_ peeks in the ``__dict__`` attribute of classes on the MRO to look for attributes. This PEP introduces a hook that classes can use to override that behavior for specific classes. Rationale ========= Peeking in the class ``__dict__`` works for regular classes, but can cause problems when a class dynamicly looks up attributes in a ``__getattribute__`` method. The new hook makes it possible to introduce the same customization for attribute lookup through the `super class`_. The superclass attribute lookup hook ==================================== In C code --------- A new slot ``tp_getattro_super`` is added to the ``PyTypeObject`` struct. The ``tp_getattro`` slot for super will call this slot when it is not ``NULL``, otherwise it will peek in the class ``tp_dict``. The slot has the following prototype:: PyObject* (*getattrosuperfunc)(PyTypeObject* tp, PyObject* self, PyObject* name); The function should perform attribute lookup for *name*, but only looking in type *tp* (which will be one of the types on the MRO for *self*) and without looking in the instance *__dict__*. The function returns ``NULL`` when the attribute cannot be found, and raises and exception. Exception other than ``AttributeError`` will cause failure of super's attribute resolution. In Python code -------------- A Python class can contain a definition for a method ``__getattribute_super__`` with the following prototype:: def __getattribute_super__(self, cls, name): pass The method should perform attribute lookup for *name* on instance *self* while only looking at *cls* (it should not look in super classes or the instance *__dict__* Alternative proposals --------------------- Reuse ``tp_getattro`` ..................... It would be nice to avoid adding a new slot, thus keeping the API simpler and easier to understand. A comment on `Issue 18181`_ asked about reusing the ``tp_getattro`` slot, that is super could call the ``tp_getattro`` slot of all methods along the MRO. AFAIK that won't work because ``tp_getattro`` will look in the instance ``__dict__`` before it tries to resolve attributes using classes in the MRO. This would mean that using ``tp_getattro`` instead of peeking the class dictionaries changes the semantics of the `super class`_. Open Issues =========== * The names of the new slot and magic method are far from settled. * I'm not too happy with the prototype for the new hook. * Should ``__getattribute_super__`` be a class method instead? References ========== * `Issue 18181`_ contains a prototype implementation The prototype uses different names than this proposal. Copyright ========= This document has been placed in the public domain. .. _`Issue 18181`: http://bugs.python.org/issue18181 .. _`super class`: http://docs.python.org/3/library/functions.html?highlight=super#super

3 9

PLY in stdlib (was cffi in stdlib)
by David Beazley 15 Jul '13

15 Jul '13

> > From: Eli Bendersky <eliben(a)gmail.com> > > I'll be the first one to admit that pycparser is almost certainly not > generally useful enough to be exposed in the stdlib. So just using it as an > implementation detail is absolutely fine. PLY is a more interesting > question, however, since PLY is somewhat more generally useful. That said, > I see all this as implementation details that shouldn't distract us from > the main point of whether cffi should be added. > Regarding the inclusion of PLY or some subcomponent of it in the standard library, it's not an entirely crazy idea in my opinion. LALR(1) parsers have been around for a long time, are generally known to anyone who's used yacc/bison, and would be useful outside the context of cffi or pycparser. PLY has also been around for about 12 years and is what I would call stable. It gets an update about every year or two, but that's about it. PLY is also relatively small--just two files and about 4300 lines of code (much of which could probably be scaled down a bit). The only downside to including PLY might be the fact that there are very few people walking around who've actually had to *implement* an LALR(1) parser generator. Some of the code for that is extremely hairy and mathematical. At this time, I don't think there are any bugs in it, but it's not the sort of thing that one wants to wander into casually. Also, there are some horrible hacks in PLY that I'd really like to get rid of, but am currently stuck with due to backwards compatibility issues. Alex Gaynor has been working on a PLY variant (RPLY) geared at RPython and which has a slightly different programming interface. I'd say if we were to go down this route, he and I should work together to put together some kind of more general "parsing.lalr" package (or similar) that cleans it up and makes it more suitable as a library for building different kinds of parsing tools on top of. Cheers, Dave

9 13

Doubt about closure/function generation in bytecode
by Andrea Griffini 15 Jul '13

15 Jul '13

Is there any semantic difference between BUILD_TUPLE 0 LOAD_CONST <code object ... > MAKE_CLOSURE 0 and LOAD_CONST <code object ...> MAKE_FUNCTION 0 ? In other words is there any difference between a function and a closure that doesn't capture anything? Is it just a speed optimization for a common case? Andrea Griffini

2 1

Why is the return value of __contains__ coerced to boolean, but that of __lt__ and the like is not?
by Ben Hoyt 15 Jul '13

15 Jul '13

I'm curious why the return value of __contains__ is coerced to True or False, whereas the return value of "normal" comparison operators like __lt__ and the like are not. The latter return the value directly without forcing it to be True or False. This makes overriding __contains__ significantly less flexible, so I'm wondering why it's designed or implemented that way. (I believe it's the cmp_outcome() function in Python/ceval.c that does this: http://hg.python.org/cpython/file/db9fe49069ed/Python/ceval.c#l4545) For example, the "peewee" ORM overloads __lt__ and the like so it can map Python expressions to SQL. But it can't do this with the "in" operator due to the result of "x in y" always returning True or False in Python. So it (ab)uses the "<<" operator to do this instead (See the peewee docs at http://peewee.readthedocs.org/en/latest/peewee/querying.html#column-lookups ). I'm sure there's a good reason for why "in" is different here, but I can't see why right now. -Ben

3 4

Summary of Python tracker Issues
by Python tracker 12 Jul '13

12 Jul '13

ACTIVITY SUMMARY (2013-07-05 - 2013-07-12) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4067 ( -5) closed 26140 (+74) total 30207 (+69) Open issues with patches: 1834 Issues opened (49) ================== #7457: Adding a read_pkg_file to DistributionMetadata http://bugs.python.org/issue7457 reopened by jason.coombs #18366: Look into using setuptools 0.8 with devinabox http://bugs.python.org/issue18366 opened by brett.cannon #18367: See if a venv setup can be used for devinabox for coverage http://bugs.python.org/issue18367 opened by brett.cannon #18368: PyOS_StdioReadline() leaks memory when realloc() fails http://bugs.python.org/issue18368 opened by christian.heimes #18369: X509 cert class for ssl module http://bugs.python.org/issue18369 opened by christian.heimes #18372: _Pickler_New() doesn't call PyObject_GC_Track(self) http://bugs.python.org/issue18372 opened by christian.heimes #18373: implement sys.get/setbyteswarningflag() http://bugs.python.org/issue18373 opened by dholth #18374: ast.parse gives wrong position (col_offset) for some BinOp-s http://bugs.python.org/issue18374 opened by Aivar.Annamaa #18375: python -m test --randseed 1234 does not randomize tests http://bugs.python.org/issue18375 opened by flox #18376: show the effective count of process when running the testsuite http://bugs.python.org/issue18376 opened by flox #18378: locale.getdefaultlocale() fails on Mac OS X with default langu http://bugs.python.org/issue18378 opened by Dmitry.Jemerov #18379: SSLSocket.getpeercert(): OCSP and CRL DP URIs http://bugs.python.org/issue18379 opened by christian.heimes #18381: unittest warnings counter http://bugs.python.org/issue18381 opened by giampaolo.rodola #18382: multiprocessing's overlapped PipeConnection issues on Windows http://bugs.python.org/issue18382 opened by wprins #18383: test_warnings modifies warnings.filters when running with "-W http://bugs.python.org/issue18383 opened by flox #18384: Add devhelp build instructions to the documentation makefile http://bugs.python.org/issue18384 opened by brouberol #18387: Add 'symbols' link to pydoc's html menu bar. http://bugs.python.org/issue18387 opened by ron_adam #18389: document that os.path.relpath does not interrogate the file sy http://bugs.python.org/issue18389 opened by jveldridge #18391: socket.fromfd()'s API is difficult or impossible to use correc http://bugs.python.org/issue18391 opened by cks #18392: Doc: PyObject_Malloc() is not documented http://bugs.python.org/issue18392 opened by haypo #18393: Gestalt() is deprecated on OSX 10.8, remove support? http://bugs.python.org/issue18393 opened by ronaldoussoren #18394: cgi.FieldStorage triggers ResourceWarning sometimes http://bugs.python.org/issue18394 opened by flox #18395: Make _Py_char2wchar() and _Py_wchar2char() public http://bugs.python.org/issue18395 opened by haypo #18396: test_signal.test_issue9324() fails on buildbot AMD64 Windows7 http://bugs.python.org/issue18396 opened by haypo #18397: Python with MinGW http://bugs.python.org/issue18397 opened by Friedrich.Spee.von.Langenfeld #18400: Minor increase to Pickle test coverage http://bugs.python.org/issue18400 opened by davidsackett #18401: Tests for pdb import ~/.pdbrc http://bugs.python.org/issue18401 opened by sptonkin #18402: Finding perl64 http://bugs.python.org/issue18402 opened by mjdorma #18404: Memory leak in gdbmmodule http://bugs.python.org/issue18404 opened by bkabrda #18405: crypt.mksalt() result has unnecessarily low entropy http://bugs.python.org/issue18405 opened by christian.heimes #18406: unicodedata.itergraphemes / str.itergraphemes / str.graphemes http://bugs.python.org/issue18406 opened by dpk #18407: Fix compiler warnings in pythoncore for Win64 http://bugs.python.org/issue18407 opened by jkloth #18408: Fixes crashes found by pyfailmalloc http://bugs.python.org/issue18408 opened by haypo #18409: IDLE Improvements: Unit test for AutoComplete.py http://bugs.python.org/issue18409 opened by philwebster #18410: IDLE Improvements: Unit test for SearchDialog.py http://bugs.python.org/issue18410 opened by philwebster #18416: Move to absolute file paths for module.__file__ http://bugs.python.org/issue18416 opened by brett.cannon #18417: urlopen() has a hidden default for its timeout argument http://bugs.python.org/issue18417 opened by barry #18418: Thread.isAlive() sometimes True after fork http://bugs.python.org/issue18418 opened by emptysquare #18421: Refactor call_with_frame() function of pyexpat.c http://bugs.python.org/issue18421 opened by haypo #18423: Document limitations on -m http://bugs.python.org/issue18423 opened by acooke #18425: IDLE Unit test for IdleHistory.py http://bugs.python.org/issue18425 opened by JayKrish #18426: Crash when extension does not use PyModule_Create() http://bugs.python.org/issue18426 opened by Padowan #18428: IDLE startup error http://bugs.python.org/issue18428 opened by mwei11 #18429: IDLE: Format Paragraph doesn't function with comment blocks http://bugs.python.org/issue18429 opened by Todd.Rovito #18430: gzip, bz2, lzma: peek advances file position of existing file http://bugs.python.org/issue18430 opened by nbargnesi #18431: Encoded words in structured headers are not decoded by the new http://bugs.python.org/issue18431 opened by r.david.murray #18432: sched modules queue property should return a list, not an iter http://bugs.python.org/issue18432 opened by rhettinger #18433: Clarify venv docs about which methods check for pre-existing p http://bugs.python.org/issue18433 opened by brett.cannon #18434: Update venv example to use setuptools http://bugs.python.org/issue18434 opened by brett.cannon Most recent 15 issues with no replies (15) ========================================== #18434: Update venv example to use setuptools http://bugs.python.org/issue18434 #18433: Clarify venv docs about which methods check for pre-existing p http://bugs.python.org/issue18433 #18431: Encoded words in structured headers are not decoded by the new http://bugs.python.org/issue18431 #18429: IDLE: Format Paragraph doesn't function with comment blocks http://bugs.python.org/issue18429 #18425: IDLE Unit test for IdleHistory.py http://bugs.python.org/issue18425 #18423: Document limitations on -m http://bugs.python.org/issue18423 #18421: Refactor call_with_frame() function of pyexpat.c http://bugs.python.org/issue18421 #18410: IDLE Improvements: Unit test for SearchDialog.py http://bugs.python.org/issue18410 #18409: IDLE Improvements: Unit test for AutoComplete.py http://bugs.python.org/issue18409 #18405: crypt.mksalt() result has unnecessarily low entropy http://bugs.python.org/issue18405 #18401: Tests for pdb import ~/.pdbrc http://bugs.python.org/issue18401 #18400: Minor increase to Pickle test coverage http://bugs.python.org/issue18400 #18395: Make _Py_char2wchar() and _Py_wchar2char() public http://bugs.python.org/issue18395 #18394: cgi.FieldStorage triggers ResourceWarning sometimes http://bugs.python.org/issue18394 #18392: Doc: PyObject_Malloc() is not documented http://bugs.python.org/issue18392 Most recent 15 issues waiting for review (15) ============================================= #18432: sched modules queue property should return a list, not an iter http://bugs.python.org/issue18432 #18430: gzip, bz2, lzma: peek advances file position of existing file http://bugs.python.org/issue18430 #18429: IDLE: Format Paragraph doesn't function with comment blocks http://bugs.python.org/issue18429 #18425: IDLE Unit test for IdleHistory.py http://bugs.python.org/issue18425 #18418: Thread.isAlive() sometimes True after fork http://bugs.python.org/issue18418 #18408: Fixes crashes found by pyfailmalloc http://bugs.python.org/issue18408 #18407: Fix compiler warnings in pythoncore for Win64 http://bugs.python.org/issue18407 #18405: crypt.mksalt() result has unnecessarily low entropy http://bugs.python.org/issue18405 #18404: Memory leak in gdbmmodule http://bugs.python.org/issue18404 #18402: Finding perl64 http://bugs.python.org/issue18402 #18400: Minor increase to Pickle test coverage http://bugs.python.org/issue18400 #18396: test_signal.test_issue9324() fails on buildbot AMD64 Windows7 http://bugs.python.org/issue18396 #18393: Gestalt() is deprecated on OSX 10.8, remove support? http://bugs.python.org/issue18393 #18389: document that os.path.relpath does not interrogate the file sy http://bugs.python.org/issue18389 #18387: Add 'symbols' link to pydoc's html menu bar. http://bugs.python.org/issue18387 Top 10 most discussed issues (10) ================================= #18408: Fixes crashes found by pyfailmalloc http://bugs.python.org/issue18408 14 msgs #10042: functools.total_ordering fails to handle NotImplemented correc http://bugs.python.org/issue10042 12 msgs #18257: Two copies of python-config http://bugs.python.org/issue18257 12 msgs #18378: locale.getdefaultlocale() fails on Mac OS X with default langu http://bugs.python.org/issue18378 10 msgs #18100: socket.sendall() cannot send buffers of 2GB or more http://bugs.python.org/issue18100 9 msgs #18365: Idle: mock Text class and test thereof http://bugs.python.org/issue18365 9 msgs #18430: gzip, bz2, lzma: peek advances file position of existing file http://bugs.python.org/issue18430 9 msgs #18389: document that os.path.relpath does not interrogate the file sy http://bugs.python.org/issue18389 8 msgs #18397: Python with MinGW http://bugs.python.org/issue18397 8 msgs #18426: Crash when extension does not use PyModule_Create() http://bugs.python.org/issue18426 8 msgs Issues closed (72) ================== #2554: test_ioctl failed Python 2.6a2 Solaris 10 SUN C http://bugs.python.org/issue2554 closed by christian.heimes #3093: Namespace pollution from multiprocessing http://bugs.python.org/issue3093 closed by christian.heimes #3266: Python-2.5.2/Modules/mmapmodule.c:915: error: `O_RDWR' undecla http://bugs.python.org/issue3266 closed by christian.heimes #3329: API for setting the memory allocator used by Python http://bugs.python.org/issue3329 closed by haypo #5060: gcc profile guided optimization http://bugs.python.org/issue5060 closed by christian.heimes #7573: Position independent include of Python.h http://bugs.python.org/issue7573 closed by christian.heimes #7686: redundant open modes 'rbb', 'wbb', 'abb' no longer work on Win http://bugs.python.org/issue7686 closed by christian.heimes #9838: Inadequate C-API to Python 3 I/O objects http://bugs.python.org/issue9838 closed by christian.heimes #11012: Add log1p(), exp1m(), gamma(), and lgamma() to cmath http://bugs.python.org/issue11012 closed by rhettinger #11366: xml.etree.ElementTree.Element: docs should mention immediate s http://bugs.python.org/issue11366 closed by christian.heimes #11908: Weird `slice.stop or sys.maxint` http://bugs.python.org/issue11908 closed by rhettinger #12115: some tests need to be skipped on threadless systems http://bugs.python.org/issue12115 closed by christian.heimes #12272: Python 2.7.1 version conflict for package "Tcl" on Windows 7 http://bugs.python.org/issue12272 closed by christian.heimes #12984: XML NamedNodeMap ( attribName in NamedNodeMap fails ) http://bugs.python.org/issue12984 closed by christian.heimes #12990: launcher can't work on path including tradition chinese char http://bugs.python.org/issue12990 closed by ronaldoussoren #13396: new method random.getrandbytes() http://bugs.python.org/issue13396 closed by rhettinger #13418: Embedded Python memory leak http://bugs.python.org/issue13418 closed by christian.heimes #13871: namedtuple does not normalize field names when checking for du http://bugs.python.org/issue13871 closed by rhettinger #14139: test_ftplib: segfault http://bugs.python.org/issue14139 closed by christian.heimes #14268: _move_file is broken because of a bad mock http://bugs.python.org/issue14268 closed by christian.heimes #14620: Fatal Python error: Cannot recover from stack overflow. http://bugs.python.org/issue14620 closed by christian.heimes #14748: spwd.getspall() is returning LDAP (non local) users too http://bugs.python.org/issue14748 closed by christian.heimes #15197: test_gettext failure on Win64 buildbot http://bugs.python.org/issue15197 closed by christian.heimes #15553: Segfault in test_6_daemon_threads() of test_threading, on Mac http://bugs.python.org/issue15553 closed by ronaldoussoren #17198: dbm.whichdb references unitialized 'ndbm' variable http://bugs.python.org/issue17198 closed by ezio.melotti #17221: Resort Misc/NEWS http://bugs.python.org/issue17221 closed by christian.heimes #17845: Clarify successful build message http://bugs.python.org/issue17845 closed by brett.cannon #17860: subprocess docs lack info how to use output result http://bugs.python.org/issue17860 closed by ronaldoussoren #17987: test.support.captured_stderr, captured_stdin not documented http://bugs.python.org/issue17987 closed by r.david.murray #18013: cgi.FieldStorage does not parse W3C sample http://bugs.python.org/issue18013 closed by flox #18020: html.escape 10x slower than cgi.escape http://bugs.python.org/issue18020 closed by ezio.melotti #18044: Email headers do not properly decode to unicode. http://bugs.python.org/issue18044 closed by r.david.murray #18101: Tk.split() doesn't work with nested Unicode strings http://bugs.python.org/issue18101 closed by serhiy.storchaka #18106: There are unused variables in Lib/test/test_collections.py http://bugs.python.org/issue18106 closed by ezio.melotti #18116: getpass.unix_getpass() always fallback to sys.stdin http://bugs.python.org/issue18116 closed by r.david.murray #18136: Put local build paths before system build paths in configure.a http://bugs.python.org/issue18136 closed by brett.cannon #18203: Replace direct calls to malloc() with PyMem_Malloc() or PyMem_ http://bugs.python.org/issue18203 closed by haypo #18231: What's new in Python should explain what's new in UCD http://bugs.python.org/issue18231 closed by belopolsky #18289: python.org Interactive interpreter linked with libedit can seg http://bugs.python.org/issue18289 closed by loewis #18308: checkRecvmsgAddress wrong in test_socket.py (AIX failures) http://bugs.python.org/issue18308 closed by neologix #18338: python --version should send output to STDOUT http://bugs.python.org/issue18338 closed by serhiy.storchaka #18342: Use the repr of a module name for ImportError in ceval.c http://bugs.python.org/issue18342 closed by brett.cannon #18351: Incorrect variable name in importlib._bootstrap._get_sourcefil http://bugs.python.org/issue18351 closed by brett.cannon #18352: collections.Counter with added attributes are not deepcopied p http://bugs.python.org/issue18352 closed by rhettinger #18354: http://www.python.org/doc/ has outdated note http://bugs.python.org/issue18354 closed by brian.curtin #18360: Won't install. Keeps telling me DLL is missing. http://bugs.python.org/issue18360 closed by terry.reedy #18361: Move dev-in-a-box to os.cpu_count() http://bugs.python.org/issue18361 closed by brett.cannon #18362: Make build_cpython.py from dev-in-a-box work outside of a box http://bugs.python.org/issue18362 closed by brett.cannon #18363: Change use of acronym tag in devinabox index.html to abbr http://bugs.python.org/issue18363 closed by brett.cannon #18364: Remove _not_found hack from importlib http://bugs.python.org/issue18364 closed by brett.cannon #18370: ast.parse produces wrong lineno and col_offset for triple-quot http://bugs.python.org/issue18370 closed by Aivar.Annamaa #18371: Should be able to specify SSL version for smtplib http://bugs.python.org/issue18371 closed by christian.heimes #18377: Python Launcher code cleanup http://bugs.python.org/issue18377 closed by ronaldoussoren #18380: email/quoprimime.py doesn't use re.sub correctly http://bugs.python.org/issue18380 closed by ezio.melotti #18385: Add devhelp build instructions to the documentation makefile ( http://bugs.python.org/issue18385 closed by ezio.melotti #18386: Better random number generator http://bugs.python.org/issue18386 closed by rhettinger #18388: Link to the contributor agreement from devinabox http://bugs.python.org/issue18388 closed by brett.cannon #18390: Better documenting of ``hg revert -r`` when merging with anoth http://bugs.python.org/issue18390 closed by brett.cannon #18398: Debug Python build fails on OS X: Fatal Python error: bad ID: http://bugs.python.org/issue18398 closed by haypo #18399: Fix grammar in comment in python.c http://bugs.python.org/issue18399 closed by r.david.murray #18403: Minor bug in The Python Tutorial http://bugs.python.org/issue18403 closed by ezio.melotti #18411: signal.SIGINT in windows cause process exit directly. http://bugs.python.org/issue18411 closed by r.david.murray #18412: Minor problem in PEP 8 http://bugs.python.org/issue18412 closed by barry #18413: logging.error('asdf', extra={'msg':123}) raise KeyError except http://bugs.python.org/issue18413 closed by christian.heimes #18414: random.choices(seq, k) http://bugs.python.org/issue18414 closed by christian.heimes #18415: Normalize single/double quote usage in importlib http://bugs.python.org/issue18415 closed by brett.cannon #18419: For reasons I can't explain Python in some cases says that get http://bugs.python.org/issue18419 closed by ezio.melotti #18420: os.rename FileNotFound complaining about dst, but it is src wh http://bugs.python.org/issue18420 closed by ned.deily #18422: is_package missing so can't use -m http://bugs.python.org/issue18422 closed by brett.cannon #18424: sum() does not return only the sum of a sequence of numbers + http://bugs.python.org/issue18424 closed by r.david.murray #18427: str.replace causes segfault for long strings http://bugs.python.org/issue18427 closed by ronaldoussoren #979407: urllib2 digest auth totally broken http://bugs.python.org/issue979407 closed by christian.heimes

1 0

Accepting PEP 445
by Antoine Pitrou 09 Jul '13

09 Jul '13

Hello, I'm accepting PEP 445 (A C API to customize memory allocators) by Victor. There is probably some grammar to correct here and there (neither Victor nor I are native English speakers), but I don't want this to hold back acceptance. The PEP is an obvious improvement in flexibility for everyone embedding Python in some external application, or even recompiling their own Python runtime for specific uses. The implementation is tracked in http://bugs.python.org/issue3329 Thanks to Victor for his patience :-) Regards Antoine.

5 7

pyfailmalloc: new debug tool to test how your applications handles MemoryError
by Victor Stinner 09 Jul '13

09 Jul '13

Hi, The PEP 445 (Add new APIs to customize Python memory allocators) has been accepted, I commited its implementation. So it's time to have fun with this API. I developed a small Python module (150 lines of C code) to inject memory allocation failures: https://pypi.python.org/pypi/pyfailmalloc This module can be used to test your application under very low memory condition, test how your application handles MemoryError. I ran it in the Python test suite: Python core and stdlib do not handle MemoryError correctly everywhere... I opened the following issue to track all bugs found by pytracemalloc and related commits to fix these bugs: http://bugs.python.org/issue18408 I already fixed bugs in PyObject_GC_NewVar(), PyDict_New(), list.pop(), type.__bases__ setter, etc. I also found a bug in _PyMem_DebugRealloc(): a function supposed to detect bugs :-) Have fun with this new tool on your applications and Python modules. Tell me if you would like to help me on issue #18408, there are still a lot of bugs to fix. Victor

1 1

Rough idea for adding introspection information for builtins
by Larry Hastings 09 Jul '13

09 Jul '13

The original impetus for Argument Clinic was adding introspection information for builtins--it seemed like any manual approach I came up with would push the builtins maintenance burden beyond the pale. Assuming that we have Argument Clinic or something like it, we don't need to optimize for ease of use from the API end--we can optimize for data size. So the approach writ large: store a blob of data associated with each entry point, as small as possible. Reconstitute the appropriate inspect.Signature on demand by reading that blob. Where to store the data? PyMethodDef is the obvious spot, but I think that structure is part of the stable ABI. So we'd need a new PyMethodDefEx and that'd be a little tiresome. Less violent to the ABI would be defining a new array of pointers-to-introspection-blobs, parallel to the PyMethodDef array, passed in via a new entry point. On to the representation. Consider the function def foo(arg, b=3, *, kwonly='a'): pass I considered four approaches, each listed below along with its total size if it was stored as C static data. 1. A specialized bytecode format, something like pickle, like this: bytes([ PARAMETER_START_LENGTH_3, 'a', 'r', 'g', PARAMETER_START_LENGTH_1, 'b', PARAMETER_DEFAULT_LENGTH_1, '3', KEYWORD_ONLY, PARAMETER_START_LENGTH_6, 'k', 'w', 'o', 'n', 'l', 'y', PARAMETER_DEFAULT_LENGTH_3, '\'', 'a', '\'', END ]) Length: 20 bytes. 2. Just use pickle--pickle the result of inspect.signature() run on a mocked-up signature, just store that. Length: 130 bytes. (Assume a two-byte size stored next to it.) 3. Store a string that, if eval'd, would produce the inspect.Signature. Length: 231 bytes. (This could be made smaller if we could assume "from inspect import *" or "p = inspect.Parameter" or something, but it'd still be easily the heaviest.) 4. Store a string that looks like the Python declaration of the signature, and parse it (Nick's suggestion). For foo above, this would be "(arg,b=3,*,kwonly='a')". Length: 23 bytes. Of those, Nick's suggestion seems best. It's slightly bigger than the specialized bytecode format, but it's human-readable (and human-writable!), and it'd be the easiest to implement. My first idea for implementation: add a "def x" to the front and ": pass" to the end, then run it through ast.parse. Iterate over the tree, converting parameters into inspect.Parameters and handling the return annotation if present. Default values and annotations would be turned into values by ast.eval_literal. (It wouldn't surprise me if there's a cleaner way to do it than the fake function definition; I'm not familiar with the ast module.) We'd want one more mild hack: the DSL will support positional parameters, and inspect.Signature supports positional parameters, so it'd be nice to render that information. But we can't represent that in Python syntax (or at least not yet!), so we can't let ast.parse see it. My suggestion: run it through ast.parse, and if it throws a SyntaxError see if the problem was a slash. If it was, remove the slash, reprocess through ast.parse, and remember that all parameters are positional-only (and barf if there are kwonly, args, or kwargs). Thoughts? //arry/

9 23