Mailman 3 May 2012 - Python-Dev

Re: [Python-Dev] cpython: Describe the default hash correctly, and mark a couple of CPython
by Antoine Pitrou May 20, 2012

May 20, 2012

On Sun, 20 May 2012 10:31:01 +0200 nick.coghlan <python-checkins(a)python.org> wrote: > + > + .. impl-detail:: > + > + CPython uses ``hash(id(x))`` as the default hash for class instances. This isn't true: >>> class C: pass ... >>> c = C() >>> hash(c) 619973 >>> id(c) 9919568 >>> hash(id(c)) 9919568 id(...) always has the lower bits clear, so it was decided to shift it to the right by a number of bits. Regards Antoine.

3 3

Re: [Python-Dev] [Python-checkins] cpython: Clean up the PCBuild project files, removing redundant settings and
by Nadeem Vawda May 19, 2012

May 19, 2012

On Sat, May 19, 2012 at 2:11 PM, kristjan.jonsson < python-checkins(a)python.org> wrote: > +Visual Studio 2010 uses version 10 of the C runtime (MSVCRT9). The > executables > Shouldn't that be MSVCRT10? Nadeem

1 0

Re: [Python-Dev] [Python-checkins] cpython: Remove outdated statements about threading and imports.
by Nick Coghlan May 18, 2012

May 18, 2012

I know you fixed the deadlock problem, but the warnings about shutdown misbehaviour are still valid. -- Sent from my phone, thus the relative brevity :) On May 18, 2012 9:59 PM, "antoine.pitrou" <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/565734c9b66d > changeset: 77020:565734c9b66d > parent: 77018:364289cc7891 > user: Antoine Pitrou <solipsis(a)pitrou.net> > date: Fri May 18 13:57:04 2012 +0200 > summary: > … [View More]Remove outdated statements about threading and imports. > > files: > Doc/library/multiprocessing.rst | 4 +-- > Doc/library/threading.rst | 23 --------------------- > 2 files changed, 1 insertions(+), 26 deletions(-) > > > diff --git a/Doc/library/multiprocessing.rst > b/Doc/library/multiprocessing.rst > --- a/Doc/library/multiprocessing.rst > +++ b/Doc/library/multiprocessing.rst > @@ -120,9 +120,7 @@ > print(q.get()) # prints "[42, None, 'hello']" > p.join() > > - Queues are thread and process safe, but note that they must never > - be instantiated as a side effect of importing a module: this can lead > - to a deadlock! (see :ref:`threaded-imports`) > + Queues are thread and process safe. > > **Pipes** > > diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst > --- a/Doc/library/threading.rst > +++ b/Doc/library/threading.rst > @@ -996,27 +996,3 @@ > Currently, :class:`Lock`, :class:`RLock`, :class:`Condition`, > :class:`Semaphore`, and :class:`BoundedSemaphore` objects may be used as > :keyword:`with` statement context managers. > - > - > -.. _threaded-imports: > - > -Importing in threaded code > --------------------------- > - > -While the import machinery is thread-safe, there are two key restrictions > on > -threaded imports due to inherent limitations in the way that > thread-safety is > -provided: > - > -* Firstly, other than in the main module, an import should not have the > - side effect of spawning a new thread and then waiting for that thread in > - any way. Failing to abide by this restriction can lead to a deadlock if > - the spawned thread directly or indirectly attempts to import a module. > -* Secondly, all import attempts must be completed before the interpreter > - starts shutting itself down. This can be most easily achieved by only > - performing imports from non-daemon threads created through the threading > - module. Daemon threads and threads created directly with the thread > - module will require some other form of synchronization to ensure they do > - not attempt imports after system shutdown has commenced. Failure to > - abide by this restriction will lead to intermittent exceptions and > - crashes during interpreter shutdown (as the late imports attempt to > - access machinery which is no longer in a valid state). > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > > [View Less]

2 2

Summary of Python tracker Issues
by Python tracker May 18, 2012

May 18, 2012

ACTIVITY SUMMARY (2012-05-11 - 2012-05-18) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3432 (+14) closed 23196 (+53) total 26628 (+67) Open issues with patches: 1457 Issues opened (52) ================== #14784: Re-importing _warnings changes warnings.filters http://bugs.python.org/issue14784 opened by brett.cannon #14785: Add sys.… [View More]

1 0

C-level duck typing
by Dag Sverre Seljebotn May 18, 2012

May 18, 2012

Hi python-dev, these ideas/questions comes out of the Cython and NumPy developer lists. What we want is a way to communicate things on the C level about the extension type instances we pass around. The solution today is often to rely on PyObject_TypeCheck. For instance, hundreds of handcrafted C extensions rely on the internal structure of NumPy arrays, and Cython will check whether objects are instances of a Cython class or not. However, this creates one-to-many situations; only one … [View More]implementor of an object API/ABI, but many consumers. What we would like is multiple implementors and multiple consumers of mutually agreed-upon standards. We essentially want more duck typing on the C level. A similar situation was PEP 3118. But there's many more such things one might want to communicate at the C level, many of which are very domain-specific and not suitable for a PEP at all. Also PEPs don't backport well to older versions of Python. What we *think* we would like (but we want other suggestions!) is an arbitrarily extensible type object, without tying this into the type hierarchy. Say you have typedef struct { unsigned long extension_id; void *data; } PyTypeObjectExtensionEntry; and then a type object can (somehow!) point to an array of these. The array is linearly scanned by consumers for IDs they recognize (most types would only have one or two entries). Cython could then get a reserved ID space to communicate whatever it wants, NumPy another one, and there could be "unofficial PEPs" where two or more projects get together to draft a spec for a particular type extension ID without having to bother python-dev about it. And, we want this to somehow work with existing Python; we still support users on Python 2.4. Options we've thought of so far: a) Use dicts and capsules to get information across. But performance-wise the dict lookup is not an option for what we want to use this for in Cython. b) Implement a metaclass which extends PyTypeObject in this way. However, that means a common runtime dependency for libraries that want to use this scheme, which is a big disadvantage to us. Today, Cython doesn't ship a runtime library but only creates standalone compileable C files, and there's no dependency from NumPy on Cython or the other way around. c) Hijack a free bit in tp_flags (22?) which we use to indicate that the PyTypeObject struct is immediately followed by a pointer to such an array. The final approach is drafted in more detail at http://wiki.cython.org/enhancements/cep1001 . To us that looks very attractive both for the speed and for the lack of runtime dependencies, and it seems like it should work in existing versions of Python. But do please feel free to tell us we are misguided. Hijacking a flag bit certainly feels dirty. Examples of how this would be used: - In Cython, we'd like to use this to annotate callable objects that happen to wrap a C function with their corresponding C function pointers. That way, callables that wrap a C function could be "unboxed", so that Cython could "cast" the Python object "scipy.special.gamma" to a function pointer at runtime and speed up the call with an order of magnitude. SciPy and Cython just needs to agree on a spec. - Lots of C extensions rely on using PyObject_TypeCheck (or even do an exact check) before calling the NumPy C API with PyArrayObject* arguments. This means that new features all have to go into NumPy; it is rather difficult to create new experimental array libraries. Extensible PyTypeObject would open up the way for other experimental array libraries; NumPy could make the standards, but others implement them (without getting NumPy as a runtime dependency, which is the consequence of subclassing). Of course, porting over the hundreds (thousands?) of extensions relying on the NumPy C API is a lot of work, but we can at least get started... Ideas? Dag Sverre Seljebotn [View Less]

9 33

Re: [Python-Dev] [Python-checkins] cpython: Issue #14624: UTF-16 decoding is now 3x to 4x faster on various inputs.
by Victor Stinner May 17, 2012

May 17, 2012

> http://hg.python.org/cpython/rev/cdcc816dea85 > changeset: 76971:cdcc816dea85 > user: Antoine Pitrou <solipsis(a)pitrou.net> > date: Tue May 15 23:48:04 2012 +0200 > summary: > Issue #14624: UTF-16 decoding is now 3x to 4x faster on various inputs. > Patch by Serhiy Storchaka. Such optimization should be mentioned in the What's New in Python 3.3 doc if Python 3.3 is now faster than Python 3.2. Same … [View More]

1 0

64-bit Windows buildbots needed
by Antoine Pitrou May 16, 2012

May 16, 2012

Hello all, We still need 64-bit Windows buildbots to test for regressions. Otherwise we might let regressions slip through, since few people seem to run the test suite under Windows at home. Regards Antoine.

2 1

Re: [Python-Dev] [Python-checkins] devguide: Add VS2010 link and text, then restructure a few things
by "Martin v. Löwis" May 16, 2012

May 16, 2012

> +All versions previous to 3.3 use Microsoft Visual Studio 2008, available at > +https://www.microsoft.com/visualstudio/en-us/products/2008-editions/express. This isn't actually the case. 2.4 and 2.5 used Visual Studio 2003, 2.0 to 2.3 used VC6, 1.4 and 1.5 used Visual C++ 1.5; versions before that were available only from Mark Hammond. Regards, Martin

2 1

sys.implementation
by Eric Snow May 15, 2012

May 15, 2012

The proposal of adding sys.implementation has come up a couple times over the last few years. [1][2] While the reaction has been overwhelmingly positive, nothing has come of it. I've created a tracker issue and a patch: http://bugs.python.org/issue14673 The patch adds a struct sequence that holds ("name" => "CPython", "version" => sys.version_info). If later needs dictate more fields, we can cross that bridge then. Are there any objections? Considering the positive reaction and … [View More]

16 51

zipimport to read from a file object instead of just a path?
by Shy Shalom May 15, 2012

May 15, 2012

In zipimport.c, function get_data(), the zip file is opened using fopen() and read with CLib functions. Did anyone ever consider making it possible to read the zipped data from a generic file object and not just using a string path? Using StringIO, This would allow a higher degree of python embedding in an application. I would be able to have a zip file in memory and make python read modules from it.

2 2