Mailman 3 May 2012 - Python-Dev

Re: [Python-Dev] cpython: Describe the default hash correctly, and mark a couple of CPython
by Antoine Pitrou 20 May '12

20 May '12

On Sun, 20 May 2012 10:31:01 +0200 nick.coghlan <python-checkins(a)python.org> wrote: > + > + .. impl-detail:: > + > + CPython uses ``hash(id(x))`` as the default hash for class instances. This isn't true: >>> class C: pass ... >>> c = C() >>> hash(c) 619973 >>> id(c) 9919568 >>> hash(id(c)) 9919568 id(...) always has the lower bits clear, so it was decided to shift it to the right by a number of bits. Regards Antoine.

3 3

Re: [Python-Dev] [Python-checkins] cpython: Clean up the PCBuild project files, removing redundant settings and
by Nadeem Vawda 20 May '12

20 May '12

On Sat, May 19, 2012 at 2:11 PM, kristjan.jonsson < python-checkins(a)python.org> wrote: > +Visual Studio 2010 uses version 10 of the C runtime (MSVCRT9). The > executables > Shouldn't that be MSVCRT10? Nadeem

1 0

Re: [Python-Dev] [Python-checkins] cpython: Remove outdated statements about threading and imports.
by Nick Coghlan 19 May '12

19 May '12

I know you fixed the deadlock problem, but the warnings about shutdown misbehaviour are still valid. -- Sent from my phone, thus the relative brevity :) On May 18, 2012 9:59 PM, "antoine.pitrou" <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/565734c9b66d > changeset: 77020:565734c9b66d > parent: 77018:364289cc7891 > user: Antoine Pitrou <solipsis(a)pitrou.net> > date: Fri May 18 13:57:04 2012 +0200 > summary: > Remove outdated statements about threading and imports. > > files: > Doc/library/multiprocessing.rst | 4 +-- > Doc/library/threading.rst | 23 --------------------- > 2 files changed, 1 insertions(+), 26 deletions(-) > > > diff --git a/Doc/library/multiprocessing.rst > b/Doc/library/multiprocessing.rst > --- a/Doc/library/multiprocessing.rst > +++ b/Doc/library/multiprocessing.rst > @@ -120,9 +120,7 @@ > print(q.get()) # prints "[42, None, 'hello']" > p.join() > > - Queues are thread and process safe, but note that they must never > - be instantiated as a side effect of importing a module: this can lead > - to a deadlock! (see :ref:`threaded-imports`) > + Queues are thread and process safe. > > **Pipes** > > diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst > --- a/Doc/library/threading.rst > +++ b/Doc/library/threading.rst > @@ -996,27 +996,3 @@ > Currently, :class:`Lock`, :class:`RLock`, :class:`Condition`, > :class:`Semaphore`, and :class:`BoundedSemaphore` objects may be used as > :keyword:`with` statement context managers. > - > - > -.. _threaded-imports: > - > -Importing in threaded code > --------------------------- > - > -While the import machinery is thread-safe, there are two key restrictions > on > -threaded imports due to inherent limitations in the way that > thread-safety is > -provided: > - > -* Firstly, other than in the main module, an import should not have the > - side effect of spawning a new thread and then waiting for that thread in > - any way. Failing to abide by this restriction can lead to a deadlock if > - the spawned thread directly or indirectly attempts to import a module. > -* Secondly, all import attempts must be completed before the interpreter > - starts shutting itself down. This can be most easily achieved by only > - performing imports from non-daemon threads created through the threading > - module. Daemon threads and threads created directly with the thread > - module will require some other form of synchronization to ensure they do > - not attempt imports after system shutdown has commenced. Failure to > - abide by this restriction will lead to intermittent exceptions and > - crashes during interpreter shutdown (as the late imports attempt to > - access machinery which is no longer in a valid state). > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > >

2 2

Summary of Python tracker Issues
by Python tracker 18 May '12

18 May '12

ACTIVITY SUMMARY (2012-05-11 - 2012-05-18) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3432 (+14) closed 23196 (+53) total 26628 (+67) Open issues with patches: 1457 Issues opened (52) ================== #14784: Re-importing _warnings changes warnings.filters http://bugs.python.org/issue14784 opened by brett.cannon #14785: Add sys._debugmallocstats() http://bugs.python.org/issue14785 opened by dmalcolm #14787: pkgutil.walk_packages returns extra modules http://bugs.python.org/issue14787 opened by cjerdonek #14788: Pdb debugs itself after ^C and a breakpoint is set anywhere http://bugs.python.org/issue14788 opened by xdegaye #14789: after continue, Pdb stops at a line without a breakpoint http://bugs.python.org/issue14789 opened by xdegaye #14790: use packaging in setup.py http://bugs.python.org/issue14790 opened by pitrou #14791: setup.py only adds /prefix/lib, not /prefix/lib64 http://bugs.python.org/issue14791 opened by pitrou #14792: setting a bp on current function, Pdb stops at next line altho http://bugs.python.org/issue14792 opened by xdegaye #14794: slice.indices raises OverflowError http://bugs.python.org/issue14794 opened by Paul.Upchurch #14795: Pdb incorrectly handles a method breakpoint when module not im http://bugs.python.org/issue14795 opened by xdegaye #14796: Calendar module test coverage improved http://bugs.python.org/issue14796 opened by Oleg.Plakhotnyuk #14797: Deprecate imp.find_module()/load_module() http://bugs.python.org/issue14797 opened by brett.cannon #14798: pyclbr raises KeyError when the prefix of a dotted name is not http://bugs.python.org/issue14798 opened by xdegaye #14799: Tkinter ttk tests hang on linux http://bugs.python.org/issue14799 opened by asvetlov #14802: Python fails to compile with VC11 ARM configuration http://bugs.python.org/issue14802 opened by Minmin.Gong #14803: Add feature to allow code execution prior to __main__ invocati http://bugs.python.org/issue14803 opened by ncoghlan #14804: Wrong defaults args notation in docs http://bugs.python.org/issue14804 opened by hynek #14805: Support display of both __cause__ and __context__ http://bugs.python.org/issue14805 opened by ncoghlan #14807: Move tarfile.filemode() into stat module http://bugs.python.org/issue14807 opened by giampaolo.rodola #14808: Pdb does not stop at a breakpoint set on the line of a functio http://bugs.python.org/issue14808 opened by xdegaye #14810: Bug in tarfile http://bugs.python.org/issue14810 opened by hwm #14811: decoding_fgets() truncates long lines and fails with a SyntaxE http://bugs.python.org/issue14811 opened by v+python #14812: Change file associations to not be a default installer feature http://bugs.python.org/issue14812 opened by brian.curtin #14813: Can't build under VS2008 anymore http://bugs.python.org/issue14813 opened by pitrou #14814: Implement PEP 3144 (the ipaddress module) http://bugs.python.org/issue14814 opened by ncoghlan #14815: random_seed uses only 32-bits of hash on Win64 http://bugs.python.org/issue14815 opened by loewis #14817: pkgutil.extend_path has no tests http://bugs.python.org/issue14817 opened by eric.smith #14818: C implementation of ElementTree: Some functions should support http://bugs.python.org/issue14818 opened by cmn #14821: Ctypes extension module builds as _ctypes_test.pyd http://bugs.python.org/issue14821 opened by jason.coombs #14822: Build unusable when compiled for Win 64-bit release http://bugs.python.org/issue14822 opened by jason.coombs #14824: reprlib documentation references string module http://bugs.python.org/issue14824 opened by magcius #14826: urllib2.urlopen fails to load URL http://bugs.python.org/issue14826 opened by wichert #14830: pysetup fails on non-ascii filenames http://bugs.python.org/issue14830 opened by tarek #14831: make r argument on itertools.combinations() optional http://bugs.python.org/issue14831 opened by djc #14833: Copyright date in footer of /pypi says 2011 http://bugs.python.org/issue14833 opened by antlong #14834: A list of broken links on the python.org website http://bugs.python.org/issue14834 opened by antlong #14835: plistlib: output empty elements correctly http://bugs.python.org/issue14835 opened by ssm #14836: Add next(iter(o)) to set.pop, dict.popitem entries. http://bugs.python.org/issue14836 opened by terry.reedy #14837: Better SSL errors http://bugs.python.org/issue14837 opened by pitrou #14838: IDLE Will not load on reinstall http://bugs.python.org/issue14838 opened by BugReporter #14840: Tutorial: Add a bit on the difference between tuples and lists http://bugs.python.org/issue14840 opened by zach.ware #14841: os.get_terminal_size() should check stdin as a fallback http://bugs.python.org/issue14841 opened by Arfrever #14842: Link to time.time() in the docs of time.localtime() is wrong http://bugs.python.org/issue14842 opened by petri.lehtinen #14843: support define_macros / undef_macros in setup.cfg http://bugs.python.org/issue14843 opened by dholth #14844: netrc does not handle accentuated characters http://bugs.python.org/issue14844 opened by drzraf #14845: list(<generator expression>) != [<list comprehension>] http://bugs.python.org/issue14845 opened by Peter.Norvig #14846: Change in error when sys.path contains a nonexistent folder (i http://bugs.python.org/issue14846 opened by takluyver #14847: AttributeError: NoneType has no attribute 'utf_8_decode' http://bugs.python.org/issue14847 opened by jason.coombs #14848: os.rename should not be used http://bugs.python.org/issue14848 opened by nvetoshkin #14849: C implementation of ElementTree: Inheriting from Element break http://bugs.python.org/issue14849 opened by cmn #14850: The inconsistency of codecs.charmap_decode http://bugs.python.org/issue14850 opened by storchaka #1635217: Warn against using requires/provides/obsoletes in setup.py http://bugs.python.org/issue1635217 reopened by techtonik Most recent 15 issues with no replies (15) ========================================== #14850: The inconsistency of codecs.charmap_decode http://bugs.python.org/issue14850 #14849: C implementation of ElementTree: Inheriting from Element break http://bugs.python.org/issue14849 #14844: netrc does not handle accentuated characters http://bugs.python.org/issue14844 #14843: support define_macros / undef_macros in setup.cfg http://bugs.python.org/issue14843 #14842: Link to time.time() in the docs of time.localtime() is wrong http://bugs.python.org/issue14842 #14841: os.get_terminal_size() should check stdin as a fallback http://bugs.python.org/issue14841 #14837: Better SSL errors http://bugs.python.org/issue14837 #14835: plistlib: output empty elements correctly http://bugs.python.org/issue14835 #14833: Copyright date in footer of /pypi says 2011 http://bugs.python.org/issue14833 #14830: pysetup fails on non-ascii filenames http://bugs.python.org/issue14830 #14814: Implement PEP 3144 (the ipaddress module) http://bugs.python.org/issue14814 #14812: Change file associations to not be a default installer feature http://bugs.python.org/issue14812 #14808: Pdb does not stop at a breakpoint set on the line of a functio http://bugs.python.org/issue14808 #14805: Support display of both __cause__ and __context__ http://bugs.python.org/issue14805 #14795: Pdb incorrectly handles a method breakpoint when module not im http://bugs.python.org/issue14795 Most recent 15 issues waiting for review (15) ============================================= #14840: Tutorial: Add a bit on the difference between tuples and lists http://bugs.python.org/issue14840 #14837: Better SSL errors http://bugs.python.org/issue14837 #14836: Add next(iter(o)) to set.pop, dict.popitem entries. http://bugs.python.org/issue14836 #14835: plistlib: output empty elements correctly http://bugs.python.org/issue14835 #14824: reprlib documentation references string module http://bugs.python.org/issue14824 #14818: C implementation of ElementTree: Some functions should support http://bugs.python.org/issue14818 #14813: Can't build under VS2008 anymore http://bugs.python.org/issue14813 #14811: decoding_fgets() truncates long lines and fails with a SyntaxE http://bugs.python.org/issue14811 #14808: Pdb does not stop at a breakpoint set on the line of a functio http://bugs.python.org/issue14808 #14807: Move tarfile.filemode() into stat module http://bugs.python.org/issue14807 #14804: Wrong defaults args notation in docs http://bugs.python.org/issue14804 #14798: pyclbr raises KeyError when the prefix of a dotted name is not http://bugs.python.org/issue14798 #14796: Calendar module test coverage improved http://bugs.python.org/issue14796 #14795: Pdb incorrectly handles a method breakpoint when module not im http://bugs.python.org/issue14795 #14792: setting a bp on current function, Pdb stops at next line altho http://bugs.python.org/issue14792 Top 10 most discussed issues (10) ================================= #14813: Can't build under VS2008 anymore http://bugs.python.org/issue14813 26 msgs #13210: Support Visual Studio 2010 http://bugs.python.org/issue13210 18 msgs #14315: zipfile.ZipFile() unable to open zip File http://bugs.python.org/issue14315 13 msgs #8271: str.decode('utf8', 'replace') -- conformance with Unicode 5.2. http://bugs.python.org/issue8271 12 msgs #14780: urllib.request could use the default CA store http://bugs.python.org/issue14780 12 msgs #11959: smtpd cannot be used without affecting global state http://bugs.python.org/issue11959 11 msgs #14807: Move tarfile.filemode() into stat module http://bugs.python.org/issue14807 11 msgs #14811: decoding_fgets() truncates long lines and fails with a SyntaxE http://bugs.python.org/issue14811 11 msgs #12029: Catching virtual subclasses in except clauses http://bugs.python.org/issue12029 10 msgs #14674: Add link to RFC 4627 from json documentation http://bugs.python.org/issue14674 10 msgs Issues closed (49) ================== #5730: setdefault speedup http://bugs.python.org/issue5730 closed by pitrou #6302: Add decode_header_as_string method to email.utils http://bugs.python.org/issue6302 closed by r.david.murray #6544: Fix refleak in kqueue implementation http://bugs.python.org/issue6544 closed by pitrou #8098: PyImport_ImportModuleNoBlock() may solve problems but causes o http://bugs.python.org/issue8098 closed by pitrou #8330: Failures seen in test_gdb on buildbots http://bugs.python.org/issue8330 closed by dmalcolm #9120: Reduce pickle size for an empty set http://bugs.python.org/issue9120 closed by loewis #9251: Test for the import lock http://bugs.python.org/issue9251 closed by pitrou #9260: A finer grained import lock http://bugs.python.org/issue9260 closed by pitrou #11051: Improve Python 3.3 startup time http://bugs.python.org/issue11051 closed by pitrou #12541: Accepting Badly formed headers in urllib HTTPBasicAuth http://bugs.python.org/issue12541 closed by orsenthil #13031: small speed-up for tarfile.py when unzipping tarballs http://bugs.python.org/issue13031 closed by rosslagerwall #14082: shutil doesn't copy extended attributes http://bugs.python.org/issue14082 closed by pitrou #14245: float rounding examples in FAQ are outdated http://bugs.python.org/issue14245 closed by mark.dickinson #14366: Supporting lzma compression in zip files http://bugs.python.org/issue14366 closed by loewis #14405: Some "Other Resources" in the sidebar are hopelessly out of da http://bugs.python.org/issue14405 closed by ezio.melotti #14417: dict RuntimeError workaround http://bugs.python.org/issue14417 closed by pitrou #14419: Faster ascii decoding http://bugs.python.org/issue14419 closed by pitrou #14543: Upgrade OpenSSL on Windows to 0.9.8u http://bugs.python.org/issue14543 closed by loewis #14584: Add gzip support to xmlrpc.server http://bugs.python.org/issue14584 closed by rosslagerwall #14624: Faster utf-16 decoder http://bugs.python.org/issue14624 closed by pitrou #14682: Backport missing errnos to 2.7 http://bugs.python.org/issue14682 closed by hynek #14692: json.loads parse_constant callback not working anymore http://bugs.python.org/issue14692 closed by hynek #14702: os.makedirs breaks under autofs directories http://bugs.python.org/issue14702 closed by hynek #14732: PEP 3121 Refactoring applied to _csv module http://bugs.python.org/issue14732 closed by pitrou #14766: Non-naive time comparison throws naive time error http://bugs.python.org/issue14766 closed by r.david.murray #14770: Minor documentation fixes http://bugs.python.org/issue14770 closed by ezio.melotti #14773: fwalk breaks on dangling symlinks http://bugs.python.org/issue14773 closed by hynek #14777: Tkinter clipboard_get() decodes characters incorrectly http://bugs.python.org/issue14777 closed by ned.deily #14779: test_buffer fails on OS X universal 64-/32-bit builds http://bugs.python.org/issue14779 closed by skrah #14781: Default to year 1 in strptime if year 0 has been specified http://bugs.python.org/issue14781 closed by r.david.murray #14786: htmlparser with tag br http://bugs.python.org/issue14786 closed by ezio.melotti #14793: broken grammar in Built-in Types doc http://bugs.python.org/issue14793 closed by sandro.tosi #14800: stat.py constant comments + docstrings http://bugs.python.org/issue14800 closed by giampaolo.rodola #14801: ssize_t where size_t expected http://bugs.python.org/issue14801 closed by pitrou #14806: re.match does not match word '{' http://bugs.python.org/issue14806 closed by ezio.melotti #14809: Add HTTP status codes introduced by RFC 6585 http://bugs.python.org/issue14809 closed by hynek #14816: compilation failed on Ubuntu shared buildbot http://bugs.python.org/issue14816 closed by pitrou #14819: Add `assertIsSubclass` and `assertNotIsSubclass` to `unittest. http://bugs.python.org/issue14819 closed by ezio.melotti #14820: socket._decref_socketios and close http://bugs.python.org/issue14820 closed by giampaolo.rodola #14823: Simplify threading.Lock.acquire() description http://bugs.python.org/issue14823 closed by r.david.murray #14825: Interactive Shell vs Executed code http://bugs.python.org/issue14825 closed by mark.dickinson #14827: IDLE crash when typing ^ character on Mac OS X http://bugs.python.org/issue14827 closed by JPEC #14828: itertools.groupby not working as expected http://bugs.python.org/issue14828 closed by petri.lehtinen #14829: test_bisect failure under 64-bit Windows http://bugs.python.org/issue14829 closed by pitrou #14832: unittest's assertItemsEqual() method gives wrong order in erro http://bugs.python.org/issue14832 closed by r.david.murray #14839: xml.sax.make_parser() returns "No parsers found" http://bugs.python.org/issue14839 closed by Arfrever #1479611: speed up function calls http://bugs.python.org/issue1479611 closed by pitrou #504152: rfc822 long header continuation broken http://bugs.python.org/issue504152 closed by r.david.murray #1440472: email.Generator is not idempotent http://bugs.python.org/issue1440472 closed by r.david.murray

1 0

C-level duck typing
by Dag Sverre Seljebotn 18 May '12

18 May '12

Hi python-dev, these ideas/questions comes out of the Cython and NumPy developer lists. What we want is a way to communicate things on the C level about the extension type instances we pass around. The solution today is often to rely on PyObject_TypeCheck. For instance, hundreds of handcrafted C extensions rely on the internal structure of NumPy arrays, and Cython will check whether objects are instances of a Cython class or not. However, this creates one-to-many situations; only one implementor of an object API/ABI, but many consumers. What we would like is multiple implementors and multiple consumers of mutually agreed-upon standards. We essentially want more duck typing on the C level. A similar situation was PEP 3118. But there's many more such things one might want to communicate at the C level, many of which are very domain-specific and not suitable for a PEP at all. Also PEPs don't backport well to older versions of Python. What we *think* we would like (but we want other suggestions!) is an arbitrarily extensible type object, without tying this into the type hierarchy. Say you have typedef struct { unsigned long extension_id; void *data; } PyTypeObjectExtensionEntry; and then a type object can (somehow!) point to an array of these. The array is linearly scanned by consumers for IDs they recognize (most types would only have one or two entries). Cython could then get a reserved ID space to communicate whatever it wants, NumPy another one, and there could be "unofficial PEPs" where two or more projects get together to draft a spec for a particular type extension ID without having to bother python-dev about it. And, we want this to somehow work with existing Python; we still support users on Python 2.4. Options we've thought of so far: a) Use dicts and capsules to get information across. But performance-wise the dict lookup is not an option for what we want to use this for in Cython. b) Implement a metaclass which extends PyTypeObject in this way. However, that means a common runtime dependency for libraries that want to use this scheme, which is a big disadvantage to us. Today, Cython doesn't ship a runtime library but only creates standalone compileable C files, and there's no dependency from NumPy on Cython or the other way around. c) Hijack a free bit in tp_flags (22?) which we use to indicate that the PyTypeObject struct is immediately followed by a pointer to such an array. The final approach is drafted in more detail at http://wiki.cython.org/enhancements/cep1001 . To us that looks very attractive both for the speed and for the lack of runtime dependencies, and it seems like it should work in existing versions of Python. But do please feel free to tell us we are misguided. Hijacking a flag bit certainly feels dirty. Examples of how this would be used: - In Cython, we'd like to use this to annotate callable objects that happen to wrap a C function with their corresponding C function pointers. That way, callables that wrap a C function could be "unboxed", so that Cython could "cast" the Python object "scipy.special.gamma" to a function pointer at runtime and speed up the call with an order of magnitude. SciPy and Cython just needs to agree on a spec. - Lots of C extensions rely on using PyObject_TypeCheck (or even do an exact check) before calling the NumPy C API with PyArrayObject* arguments. This means that new features all have to go into NumPy; it is rather difficult to create new experimental array libraries. Extensible PyTypeObject would open up the way for other experimental array libraries; NumPy could make the standards, but others implement them (without getting NumPy as a runtime dependency, which is the consequence of subclassing). Of course, porting over the hundreds (thousands?) of extensions relying on the NumPy C API is a lot of work, but we can at least get started... Ideas? Dag Sverre Seljebotn

9 33

Re: [Python-Dev] [Python-checkins] cpython: Issue #14624: UTF-16 decoding is now 3x to 4x faster on various inputs.
by Victor Stinner 17 May '12

17 May '12

> http://hg.python.org/cpython/rev/cdcc816dea85 > changeset: 76971:cdcc816dea85 > user: Antoine Pitrou <solipsis(a)pitrou.net> > date: Tue May 15 23:48:04 2012 +0200 > summary: > Issue #14624: UTF-16 decoding is now 3x to 4x faster on various inputs. > Patch by Serhiy Storchaka. Such optimization should be mentioned in the What's New in Python 3.3 doc if Python 3.3 is now faster than Python 3.2. Same remark for the UTF-8 optimization. Victor

1 0

64-bit Windows buildbots needed
by Antoine Pitrou 17 May '12

17 May '12

Hello all, We still need 64-bit Windows buildbots to test for regressions. Otherwise we might let regressions slip through, since few people seem to run the test suite under Windows at home. Regards Antoine.

2 1

Re: [Python-Dev] [Python-checkins] devguide: Add VS2010 link and text, then restructure a few things
by "Martin v. Löwis" 16 May '12

16 May '12

> +All versions previous to 3.3 use Microsoft Visual Studio 2008, available at > +https://www.microsoft.com/visualstudio/en-us/products/2008-editions/express. This isn't actually the case. 2.4 and 2.5 used Visual Studio 2003, 2.0 to 2.3 used VC6, 1.4 and 1.5 used Visual C++ 1.5; versions before that were available only from Mark Hammond. Regards, Martin

2 1

sys.implementation
by Eric Snow 15 May '12

15 May '12

The proposal of adding sys.implementation has come up a couple times over the last few years. [1][2] While the reaction has been overwhelmingly positive, nothing has come of it. I've created a tracker issue and a patch: http://bugs.python.org/issue14673 The patch adds a struct sequence that holds ("name" => "CPython", "version" => sys.version_info). If later needs dictate more fields, we can cross that bridge then. Are there any objections? Considering the positive reaction and the scope of the addition, does this need a PEP? -eric [1] http://mail.python.org/pipermail/python-dev/2009-October/092893.html [2] http://mail.python.org/pipermail/python-ideas/2012-April/014878.html

16 51

zipimport to read from a file object instead of just a path?
by Shy Shalom 15 May '12

15 May '12

In zipimport.c, function get_data(), the zip file is opened using fopen() and read with CLib functions. Did anyone ever consider making it possible to read the zipped data from a generic file object and not just using a string path? Using StringIO, This would allow a higher degree of python embedding in an application. I would be able to have a zip file in memory and make python read modules from it.

2 2