Mailman 3 February 2013 - Python-Dev

Status of the PEP 433?
by Victor Stinner Feb. 14, 2013

Feb. 14, 2013

Hi, I read again all emails related to PEP 433. There are different opposition to this change and I tried to make a summary. My question is: how can I move this PEP forward (accept, defer or reject it)? Can I accept it myself, or do we need a global agreement, or should someone else decide for this PEP (who? Antoine maybe)? -- Most complains concern issues if cloexec is set by default. I don't understand why so many people feel concerned because my PEP proposes *to not change anything to … [View More]the default behaviour*: cloexec will not be set by default, as it was always the case since Python 1. You have to explicitly use -e command line option, PYTHONCLOEXEC=1 environment variable, or call sys.setdefaultcloexec(True). If you *explicitly* change the default value of the cloexec parameter, you must expect to change the behaviour of your application and maybe discover "bugs" in your application (application incompatible with cloexec set by default). (I don't think that any library would like to play with sys.setdefaultcloexec(), since most developers agree that it might break applications.) -- Ok, let's see all oppositions, presented as a FAQ. (I hope that I didn't misuse email quotes.) Q: the PEP does not solve file descriptor inherance after fork() A: Correct, it only solves issues with exec(), as explained in the PEP. Q: the PEP changes the POSIX behaviour. A: Antoine replied that "Python is not POSIX", Nick wrote "While many of Python's APIs are heavily inspired by POSIX, it still isn't POSIX" (he was talking about integer division). Perl sets cloexec flag by default since its first version, something like 25 years ago, except for explicit POSIX::* calls. Ruby 2 will also set cloexec flag by default. Q: The PEP may break applications. A: Most developers agree that it is (very) unlikely. If file descriptor inherance matters, subprocess must be used (because it rocks!) with pass_fds. If subprocess is used without pass_fds, applications stops working since python 3.2 (since python 3.0 on Windows). pass_fds will clear the close-on-exec flag on the file descriptors with the PEP. If an application breaks because it relies on file descriptor inherance, it's easy to detect it (EBADF error) and simple to fix it (ex: just call sys.setdefaultcloexec(False) at startup, or set explicitly cloexec parameter on the few functions causing trouble). Q: Performance overhead if cloexec is set by default (extra syscalls). A: The PEP now contains a benchmark: the overhead *on a microbenchmark* is between 1 and 3%. I bet that nobody will be able to notice a difference on a real application. Q: The default value of cloexec must not be configurable: "libraries cannot rely on it and have to make file descriptors cloexec on an individual basis, since the default flag can be disabled". A: IMO inherance of file descriptor doesn't matter in most cases, so only a few function must be modified to explicitly set or clear cloexec flag. Said differently: libraries should not care of the default value of the cloexec parameter, which should only be configured by applications (not libraries). Q: The default value of cloexec must not be configurable: "just by looking at this code, you have no way to know whether the file descriptor will be inherited by the child process." A: Guido wrote "Honestly, what happened to the idea that we're all adults here? The likelihood that someone is clueless enough to misuse the flag and yet clueful enough to find out how to change it seems remote -- and they pretty much deserve what they get." -- Another alternative (not listed in the PEP yet) is to leave the posix module (its name depends on the platform: posix, nt or ce) unchanged, and only modify the os module to add cloexec parameter. I implemented this alternative in the "posixmod" branch of the PEP 433 repository. It gives a better traceback on error because the cloexec flag is set in Python, so it's possible to know if the error comes from the creation of the file descriptor or setting the flag in the traceback. (Does it really matter? I never seen an error on setting or clearing the flag.) The problem is that it only concernes posix and os modules: the PEP does also modify socket, io and select modules, which don't have "low" and "high" level modules. So it's possible possible to create a socket using the POSIX behaviour. (Does it matter, it was said that Python is not POSIX.) I'm not convinced that we should do something special for posix/os, I prefer the actual version of the PEP (modify also posix). Victor [View Less]

2 1

Question regarding: Lib/_markupbase.py
by Developer Developer Feb. 14, 2013

Feb. 14, 2013

Hello all, I was having a look at the file: Lib/_markupbase.py (@ 82151), function: "_parse_doctype_element" and have seen something that has caught my attention: if '>' in rawdata[j:]: return rawdata.find(">", j) + 1 Wouldn't it be better to do the following? pos = rawdata.find(">", j) if pos != -1: return pos + 1 Otherwise I think we are scanning rawdata[j:] twice. Best regards

6 8

efficient string concatenation (yep, from 2004)
by Christian Tismer Feb. 14, 2013

Feb. 14, 2013

Hi friends, _efficient string concatenation_ has been a topic in 2004. Armin Rigo proposed a patch with the name of the subject, more precisely: /[Patches] [ python-Patches-980695 ] efficient string concatenation// //on sourceforge.net, on 2004-06-28.// / This patch was finally added to Python 2.4 on 2004-11-30. Some people might remember the larger discussion if such a patch should be accepted at all, because it changes the programming style for many of us from "don't do that, stupid" to "… [View More]well, you may do it in CPython", which has quite some impact on other implementations (is it fast on Jython, now?). It changed for instance my programming and teaching style a lot, of course! But I think nobody but people heavily involved in PyPy expected this: Now, more than eight years after that patch appeared and made it into 2.4, PyPy (!) still does _not_ have it! Obviously I was mislead by other optimizations, and the fact that this patch was from a/the major author of PyPy who invented the initial patch for CPython. That this would be in PyPy as well sooner or later was without question for me. Wrong... ;-) Yes, I agree that for PyPy it is much harder to implement without the refcounting trick, and probably even more difficult in case of the JIT. But nevertheless, I tried to find any reference to this missing crucial optimization, with no success after an hour (*). And I guess many other people are stepping in the same trap. So I can imagine that PyPy looses some of its speed in many programs, because Armin's great hack did not make it into PyPy, and this is not loudly declared somewhere. I believe the efficiency of string concatenation is something that people assume by default and add it to the vague CPython compatibility claim, if not explicitly told otherwise. ---- Some silly proof, using python 2.7.3 vs PyPy 1.9: > $ cat strconc.py > #!env python > > from timeit import default_timer as timer > > tim = timer() > > s = '' > for i in xrange(100000): > s += 'X' > > tim = timer() - tim > > print 'time for {} concats = {:0.3f}'.format(len(s), tim) > $ python strconc.py > time for 100000 concats = 0.028 > $ pypy strconc.py > time for 100000 concats = 0.804 Something is needed - a patch for PyPy or for the documentation I guess. This is not just some unoptimized function in some module, but it is used all over the place and became a very common pattern since introduced. /How ironic that a foreseen problem occurs _now_, and _there_ :-)// / cheers -- chris (*) http://pypy.readthedocs.org/en/latest/cpython_differences.html http://pypy.org/compat.html http://pypy.org/performance.html -- Christian Tismer :^) <mailto:tismer@stackless.com> Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ [View Less]

8 14

Marking GC details as CPython-only
by Maciej Fijalkowski Feb. 14, 2013

Feb. 14, 2013

Hi I've tried (and failed) to find what GC details (especially finalizer semantics) are CPython only and which ones are not. The best I could find was the documentation of __del__ here: http://docs.python.org/2/reference/datamodel.html Things were pypy differs: * finalizers in pypy will be called only once, even if the object is resurrected. I'm not sure if this is detail or we're just plain incompatible. * pypy breaks cycles and runs finalizers in random order (but topologically correct), … [View More]

6 9

Usage of += on strings in loops in stdlib
by Maciej Fijalkowski Feb. 14, 2013

Feb. 14, 2013

Hi We recently encountered a performance issue in stdlib for pypy. It turned out that someone commited a performance "fix" that uses += for strings instead of "".join() that was there before. Now this hurts pypy (we can mitigate it to some degree though) and possible Jython and IronPython too. How people feel about generally not having += on long strings in stdlib (since the refcount = 1 thing is a hack)? What about other performance improvements in stdlib that are problematic for pypy or … [View More]

21 61

what is a dict_keys and where can I import it from?
by Chris Withers Feb. 12, 2013

Feb. 12, 2013

Hi all, So, dicts in Python 3 return "something different" from their keys and values methods: >>> dict(x=1, y=2).keys() dict_keys(['y', 'x']) >>> type(dict(x=1, y=2).keys()) <class 'dict_keys'> I have vague memories of these things being referred to as views or some such? Where can I learn more? More importantly, how can I tell if I have one of them? I guess I can keep a reference to type({}.keys()) somewhere, but that feels a bit yucky. Should these things be … [View More]

4 3

Submitting PEP 422 (Simple class initialization hook) for pronouncement
by Nick Coghlan Feb. 12, 2013

Feb. 12, 2013

I wrote PEP 422 (simple class initialization hook) last year after PJE pointed out that Python 3 style metaclasses made some Python 2 code impossible to migrate (since the class body could no longer request modifications to be made to the class after initialization was complete). It then languished, as I never found the time to actually implement it. Fortunately, Daniel Urban has now agreed to be a co-author on the PEP, and has fixed a couple of lingering technical errors in the PEP, as well … [View More]as providing a reference implementation at http://bugs.python.org/issue17044 For those that don't recall the original discussion, the proposal is to add a new __init_class__ hook, invoked after the class object is created, but before the class decorators are applied. This provides a simple approach to inherited post-creation modification of classes, without the need for a custom metaclass. There's currently no BDFL delegate assigned, so this is a request for a pronouncement from Guido. Regards, Nick. ======================================= PEP: 422 Title: Simple class initialisation hook Version: $Revision$ Last-Modified: $Date$ Author: Nick Coghlan <ncoghlan(a)gmail.com>, Daniel Urban <urban.dani+py(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 5-Jun-2012 Python-Version: 3.4 Post-History: 5-Jun-2012, 10-Feb-2012 Abstract ======== In Python 2, the body of a class definition could modify the way a class was created (or simply arrange to run other code after the class was created) by setting the ``__metaclass__`` attribute in the class body. While doing this implicitly from called code required the use of an implementation detail (specifically, ``sys._getframes()``), it could also be done explicitly in a fully supported fashion (for example, by passing ``locals()`` to a function that calculated a suitable ``__metaclass__`` value) There is currently no corresponding mechanism in Python 3 that allows the code executed in the class body to directly influence how the class object is created. Instead, the class creation process is fully defined by the class header, before the class body even begins executing. This PEP proposes a mechanism that will once again allow the body of a class definition to more directly influence the way a class is created (albeit in a more constrained fashion), as well as replacing some current uses of metaclasses with a simpler, easier to understand alternative. Background ========== For an already created class ``cls``, the term "metaclass" has a clear meaning: it is the value of ``type(cls)``. *During* class creation, it has another meaning: it is also used to refer to the metaclass hint that may be provided as part of the class definition. While in many cases these two meanings end up referring to one and the same object, there are two situations where that is not the case: * If the metaclass hint refers to an instance of ``type``, then it is considered as a candidate metaclass along with the metaclasses of all of the parents of the class being defined. If a more appropriate metaclass is found amongst the candidates, then it will be used instead of the one given in the metaclass hint. * Otherwise, an explicit metaclass hint is assumed to be a factory function and is called directly to create the class object. In this case, the final metaclass will be determined by the factory function definition. In the typical case (where the factory functions just calls ``type``, or, in Python 3.3 or later, ``types.new_class``) the actual metaclass is then determined based on the parent classes. It is notable that only the actual metaclass is inherited - a factory function used as a metaclass hook sees only the class currently being defined, and is not invoked for any subclasses. In Python 3, the metaclass hint is provided using the ``metaclass=Meta`` keyword syntax in the class header. This allows the ``__prepare__`` method on the metaclass to be used to create the ``locals()`` namespace used during execution of the class body (for example, specifying the use of ``collections.OrderedDict`` instead of a regular ``dict``). In Python 2, there was no ``__prepare__`` method (that API was added for Python 3 by PEP 3115). Instead, a class body could set the ``__metaclass__`` attribute, and the class creation process would extract that value from the class namespace to use as the metaclass hint. There is `published code`_ that makes use of this feature. Another new feature in Python 3 is the zero-argument form of the ``super()`` builtin, introduced by PEP 3135. This feature uses an implicit ``__class__`` reference to the class being defined to replace the "by name" references required in Python 2. Just as code invoked during execution of a Python 2 metaclass could not call methods that referenced the class by name (as the name had not yet been bound in the containing scope), similarly, Python 3 metaclasses cannot call methods that rely on the implicit ``__class__`` reference (as it is not populated until after the metaclass has returned control to the class creation machiner). Proposal ======== This PEP proposes that a mechanism be added to Python 3 that meets the following criteria: 1. Restores the ability for class namespaces to have some influence on the class creation process (above and beyond populating the namespace itself), but potentially without the full flexibility of the Python 2 style ``__metaclass__`` hook 2. Integrates nicely with class inheritance structures (including mixins and multiple inheritance) 3. Integrates nicely with the implicit ``__class__`` reference and zero-argument ``super()`` syntax introduced by PEP 3135 4. Can be added to an existing base class without a significant risk of introducing backwards compatibility problems One mechanism that can achieve this goal is to add a new class initialisation hook, modelled directly on the existing instance initialisation hook, but with the signature constrained to match that of an ordinary class decorator. Specifically, it is proposed that class definitions be able to provide a class initialisation hook as follows:: class Example: @classmethod def __init_class__(cls): # This is invoked after the class is created, but before any # explicit decorators are called # The usual super() mechanisms are used to correctly support # multiple inheritance. The class decorator style signature helps # ensure that invoking the parent class is as simple as possible. If present on the created object, this new hook will be called by the class creation machinery *after* the ``__class__`` reference has been initialised. For ``types.new_class()``, it will be called as the last step before returning the created class object. If a metaclass wishes to block class initialisation for some reason, it must arrange for ``cls.__init_class__`` to trigger ``AttributeError``. Note, that when ``__init_class__`` is called, the name of the class is not bound to the new class object yet. As a consequence, the two argument form of ``super()`` cannot be used to call methods (e.g., ``super(Example, cls)`` wouldn't work in the example above). However, the zero argument form of ``super()`` works as expected, since the ``__class__`` reference is already initialised. This general proposal is not a new idea (it was first suggested for inclusion in the language definition `more than 10 years ago`_, and a similar mechanism has long been supported by `Zope's ExtensionClass`_), but the situation has changed sufficiently in recent years that the idea is worth reconsidering. Key Benefits ============ Replaces many use cases for dynamic setting of ``__metaclass__`` ----------------------------------------------------------------- For use cases that don't involve completely replacing the defined class, Python 2 code that dynamically set ``__metaclass__`` can now dynamically set ``__init_class__`` instead. For more advanced use cases, introduction of an explicit metaclass (possibly made available as a required base class) will still be necessary in order to support Python 3. Easier inheritance of definition time behaviour ----------------------------------------------- Understanding Python's metaclasses requires a deep understanding of the type system and the class construction process. This is legitimately seen as challenging, due to the need to keep multiple moving parts (the code, the metaclass hint, the actual metaclass, the class object, instances of the class object) clearly distinct in your mind. Even when you know the rules, it's still easy to make a mistake if you're not being extremely careful. An earlier version of this PEP actually included such a mistake: it stated "subclass of type" for a constraint that is actually "instance of type". Understanding the proposed class initialisation hook only requires understanding decorators and ordinary method inheritance, which isn't quite as daunting a task. The new hook provides a more gradual path towards understanding all of the phases involved in the class definition process. Reduced chance of metaclass conflicts ------------------------------------- One of the big issues that makes library authors reluctant to use metaclasses (even when they would be appropriate) is the risk of metaclass conflicts. These occur whenever two unrelated metaclasses are used by the desired parents of a class definition. This risk also makes it very difficult to *add* a metaclass to a class that has previously been published without one. By contrast, adding an ``__init_class__`` method to an existing type poses a similar level of risk to adding an ``__init__`` method: technically, there is a risk of breaking poorly implemented subclasses, but when that occurs, it is recognised as a bug in the subclass rather than the library author breaching backwards compatibility guarantees. In fact, due to the constrained signature of ``__init_class__``, the risk in this case is actually even lower than in the case of ``__init__``. Integrates cleanly with \PEP 3135 --------------------------------- Unlike code that runs as part of the metaclass, code that runs as part of the new hook will be able to freely invoke class methods that rely on the implicit ``__class__`` reference introduced by PEP 3135, including methods that use the zero argument form of ``super()``. Alternatives ============ The Python 3 Status Quo ----------------------- The Python 3 status quo already offers a great deal of flexibility. For changes which only affect a single class definition and which can be specified at the time the code is written, then class decorators can be used to modify a class explicitly. Class decorators largely ignore class inheritance and can make full use of methods that rely on the ``__class__`` reference being populated. Using a custom metaclass provides the same level of power as it did in Python 2. However, it's notable that, unlike class decorators, a metaclass cannot call any methods that rely on the ``__class__`` reference, as that reference is not populated until after the metaclass constructor returns control to the class creation code. One major use case for metaclasses actually closely resembles the use of class decorators. It occurs whenever a metaclass has an implementation that uses the following pattern:: class Metaclass(type): def __new__(meta, *args, **kwds): cls = super(Metaclass, meta).__new__(meta, *args, **kwds) # Do something with cls return cls The key difference between this pattern and a class decorator is that it is automatically inherited by subclasses. However, it also comes with a major disadvantage: Python does not allow you to inherit from classes with unrelated metaclasses. Thus, the status quo requires that developers choose between the following two alternatives: * Use a class decorator, meaning that behaviour is not inherited and must be requested explicitly on every subclass * Use a metaclass, meaning that behaviour is inherited, but metaclass conflicts may make integration with other libraries and frameworks more difficult than it otherwise would be If this PEP is ultimately rejected, then this is the existing design that will remain in place by default. Restoring the Python 2 metaclass hook ------------------------------------- One simple alternative would be to restore support for a Python 2 style ``metaclass`` hook in the class body. This would be checked after the class body was executed, potentially overwriting the metaclass hint provided in the class header. The main attraction of such an approach is that it would simplify porting Python 2 applications that make use of this hook (especially those that do so dynamically). However, this approach does nothing to simplify the process of adding *inherited* class definition time behaviour, nor does it interoperate cleanly with the PEP 3135 ``__class__`` and ``super()`` semantics (as with any metaclass based solution, the ``__metaclass__`` hook would have to run before the ``__class__`` reference has been populated. Dynamic class decorators ------------------------ The original version of this PEP was called "Dynamic class decorators" and focused solely on a significantly more complicated proposal than that presented in the current version. As with the current version, it proposed that a new step be added to the class creation process, after the metaclass invocation to construct the class instance and before the application of lexical decorators. However, instead of a simple process of calling a single class method that relies on normal inheritance mechanisms, it proposed a far more complicated procedure that walked the class MRO looking for decorators stored in iterable ``__decorators__`` attributes. Using the current version of the PEP, the scheme originally proposed could be implemented as:: class DynamicDecorators(Base): @classmethod def __init_class__(cls): # Process any classes later in the MRO try: mro_chain = super().__init_class__ except AttributeError: pass else: mro_chain() # Process any __decorators__ attributes in the MRO for entry in reversed(cls.mro()): decorators = entry.__dict__.get("__decorators__", ()) for deco in reversed(decorators): cls = deco(cls) Any subclasses of ``DynamicDecorators`` would then automatically have the contents of any ``__decorators__`` attributes processed and invoked. The mechanism in the current PEP is considered superior, as many issues to do with ordering and the same decorator being invoked multiple times just go away, as that kind of thing is taken care of through the use of an ordinary class method invocation. Automatic metaclass derivation ------------------------------ When no appropriate metaclass is found, it's theoretically possible to automatically derive a metaclass for a new type based on the metaclass hint and the metaclasses of the bases. While adding such a mechanism would reduce the risk of spurious metaclass conflicts, it would do nothing to improve integration with PEP 3135, would not help with porting Python 2 code that set ``__metaclass__`` dynamically and would not provide a more straightforward inherited mechanism for invoking additional operations after the class invocation is complete. In addition, there would still be a risk of metaclass conflicts in cases where the base metaclasses were not written with multiple inheritance in mind. In such situations, there's a chance of introducing latent defects if one or more metaclasses are not invoked correctly. Calling the new hook from ``type.__init__`` ------------------------------------------- Calling the new hook automatically from ``type.__init__``, would achieve most of the goals of this PEP. However, using that approach would mean that ``__init_class__`` implementations would be unable to call any methods that relied on the ``__class__`` reference (or used the zero-argument form of ``super()``), and could not make use of those features themselves. Reference Implementation ======================== The reference implementation has been posted to the `issue tracker`_. References ========== .. _published code: http://mail.python.org/pipermail/python-dev/2012-June/119878.html .. _more than 10 years ago: http://mail.python.org/pipermail/python-dev/2001-November/018651.html .. _Zope's ExtensionClass: http://docs.zope.org/zope_secrets/extensionclass.html .. _issue tracker: http://bugs.python.org/issue17044 Copyright ========= This document has been placed in the public domain. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia [View Less]

10 25

My CLA
by anatoly techtonik Feb. 11, 2013

Feb. 11, 2013

Python Contributor Agreement ---------------------------- I allow PSF to release all my code that I submitted to it, under any open source license. -- anatoly t.

7 9

checking what atexit handlers are registered in Python 3
by Chris Withers Feb. 11, 2013

Feb. 11, 2013

Hi All, I'm finally getting around to porting some of the packages I maintain over to Python 3. One rough edge I've hit: I see the atexit module has moved to be C-based and, as far as I can tell, no longer allows you to introspect what atexit functions have been registered. If I'm writing tests for code that registers atexit handlers, how can I check that they've been correctly registered? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting … [View More]

2 1

Re: [Python-Dev] [Python-checkins] cpython (3.3): Reject float as uid or gid.
by Eric V. Smith Feb. 11, 2013

Feb. 11, 2013

On 2/10/2013 4:29 PM, serhiy.storchaka wrote: > http://hg.python.org/cpython/rev/4ef048f4834e > changeset: 82147:4ef048f4834e > branch: 3.3 > parent: 82145:b322655a4a88 > user: Serhiy Storchaka <storchaka(a)gmail.com> > date: Sun Feb 10 23:28:02 2013 +0200 > summary: > Reject float as uid or gid. > A regression was introduced in the commit for issue issue #4591. > > files: > Modules/posixmodule.c | 16 ++++++++++++++-- … [View More]

2 1

_not_found attribute on ImportError
by Chris Withers Feb. 11, 2013

Feb. 11, 2013

Hi All, I see in Python 3, some ImportErrors have grown a '_not_found' attribute. What's the significance of this attribute and where/how is it added? The only way I can seem to create this attribute is: ex = ImportError ex._not_found = True cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

4 5

Re: [Python-Dev] [Python-checkins] Daily reference leaks (b53b029895df): sum=2
by Eli Bendersky Feb. 11, 2013

Feb. 11, 2013

On Sun, Feb 10, 2013 at 9:00 PM, <solipsis(a)pitrou.net> wrote: > results for b53b029895df on branch "default" > -------------------------------------------- > > test_concurrent_futures leaked [2, 1, -1] memory blocks, sum=2 > > When did this start happening? Eli

1 0

Why does Signature.from_function() have to check the type of its argument?
by Stefan Behnel Feb. 10, 2013

Feb. 10, 2013

Hi, I'm wondering about the purpose of this code in inspect.Signature.from_function(): """ if not isinstance(func, types.FunctionType): raise TypeError('{!r} is not a Python function'.format(func)) """ Is there any reason why this method would have to explicitly check the type of its argument? Why can't it just accept any object that quacks like a function? The reason why I ran into this is that I was trying to implement PEP 362 for Cython functions (which are not of Python … [View More]

5 10

os.path.join failure mode
by Thomas Scrace Feb. 10, 2013

Feb. 10, 2013

If a function (or other non-string object) is accidentally passed as an argument to os.path.join() the result is an AttributeError: In [3]: os.path.join(fn, "path") > --------------------------------------------------------------------------- > AttributeError Traceback (most recent call last) > /home/tom/<ipython-input-3-44b097ceab04> in <module>() > ----> 1 os.path.join(fn, "path") > /usr/lib/python2.7/posixpath.pyc in join(a, *p) > … [View More]

7 11

Force to use Py_CLEAR and Py_VISIT in example for Python extending
by Andrew Svetlov Feb. 9, 2013

Feb. 9, 2013

For now http://docs.python.org/3/extending/newtypes.html#supporting-cyclic-garbage-… at first the doc demonstrate long way than note Py_CLEAR and Py_VISIT macroses. I like to remove code similar to if (self->first) { vret = visit(self->first, arg); if (vret != 0) return vret; } and tmp = self->first; self->first = NULL; Py_XDECREF(tmp); and replace those to Py_VISIT and Py_CLEAN. I think we have to demonstrate best practices in … [View More]

3 2

why do we allow this syntax?
by Chris Withers Feb. 8, 2013

Feb. 8, 2013

Hi All, Just had a bit of an embarrassing incident in some code where I did: sometotal =+ somevalue I'm curious why this syntax is allowed? I'm sure there are good reasons, but thought I'd ask... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

8 12

Summary of Python tracker Issues
by Python tracker Feb. 8, 2013

Feb. 8, 2013

ACTIVITY SUMMARY (2013-02-01 - 2013-02-08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3847 ( -1) closed 25088 (+60) total 28935 (+59) Open issues with patches: 1674 Issues opened (45) ================== #13886: readline-related test_builtin failure http://bugs.python.org/issue13886 reopened by nadeem.vawda #17102: tarfile extract can write … [View More]

1 0

help with knowledge on how to find which release a fix will land in
by Chris Withers Feb. 8, 2013

Feb. 8, 2013

Hi All, Where would I look to find out which release a fix for an issue (http://bugs.python.org/issue15822 if you're interested ;-)) will land in? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

4 6

PyConTW is calling for proposals!
by LittleQ Feb. 7, 2013

Feb. 7, 2013

(link: PyConTW is calling for proposals!) http://tw.pycon.org/2013/en/blog/2012/11/21/call-for-proposals-en/ Hello, everyone! I'm Colin Su (LittleQ), Public Relation @ PyConTW. PyCon Taiwan is calling for proposals now, Python is becoming one of famous programming languages in Taiwan, We need every Python developers' participating. Go to the link above, and submit your proposal right now! :D Feel free to send me an email if you have any question. Have a nice Day, Colin, PR @ PyConTW (if … [View More]

2 1

PyCon Tickets Almost Sold Out
by Brian Curtin Feb. 7, 2013

Feb. 7, 2013

Since the Language Summit is held at PyCon I think this counts as on-topic... If you're interested in going to the conference, there are under 50 tickets remaining: https://us.pycon.org/2013/registration/

2 1

Re: [Python-Dev] cpython (2.7): - Issue #17086: Backport thepatches from the 3.3 branch to cross-build
by a.cavallo＠cavallinux.eu Feb. 7, 2013

Feb. 7, 2013

> (btw, nothing to do with this discussion, Nick, but it appears your > RHEL buildbot is offline) > > Regards Btw rpm build and packaging can be done on the suse build service: it is a very complete (and free) service. https://build.opensuse.org

1 0

Re: [Python-Dev] cpython (2.7): - Issue #17086: Backport the patches from the 3.3 branch to cross-build
by Antoine Pitrou Feb. 7, 2013

Feb. 7, 2013

On Thu, 31 Jan 2013 23:52:27 +0100 (CET) matthias.klose <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/8ee6d96a1019 > changeset: 81859:8ee6d96a1019 > branch: 2.7 > parent: 81855:df9f8feb7444 > user: doko(a)python.org > date: Thu Jan 31 23:52:03 2013 +0100 > summary: > - Issue #17086: Backport the patches from the 3.3 branch to cross-build > the package. You aren't supposed to add new features to bugfix … [View More]

7 13

PEP-0431 update 4
by Lennart Regebro Feb. 7, 2013

Feb. 7, 2013

Here is the update of PEP-0431. The major changes are that the new class now has a lowercase name, to conform to the rest of the datetime modeule: dsttimezone (i decided against dst_timezone, as non of the other classes have underscores). Also the timezone name sets are gone. Thanks to Nick Coghlan for finding a whole bunch of minor mistakes. //Lennart ---- PEP: 431 Title: Time zone support improvements Version: $Revision$ Last-Modified: $Date$ Author: Lennart Regebro <regebro(a)gmail.… [View More]com> BDFL-Delegate: Barry Warsaw <barry(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 11-Dec-2012 Post-History: 11-Dec-2012, 28-Dec-2012, 28-Jan-2013 Abstract ======== This PEP proposes the implementation of concrete time zone support in the Python standard library, and also improvements to the time zone API to deal with ambiguous time specifications during DST changes. Proposal ======== Concrete time zone support -------------------------- The time zone support in Python has no concrete implementation in the standard library outside of a tzinfo baseclass that supports fixed offsets. To properly support time zones you need to include a database over all time zones, both current and historical, including daylight saving changes. But such information changes frequently, so even if we include the last information in a Python release, that information would be outdated just a few months later. Time zone support has therefore only been available through two third-party modules, ``pytz`` and ``dateutil``, both who include and wrap the "zoneinfo" database. This database, also called "tz" or "The Olsen database", is the de-facto standard time zone database over time zones, and it is included in most Unix and Unix-like operating systems, including OS X. This gives us the opportunity to include the code that supports the zoneinfo data in the standard library, but by default use the operating system's copy of the data, which typically will be kept updated by the updating mechanism of the operating system or distribution. For those who have an operating system that does not include the zoneinfo database, for example Windows, the Python source distribution will include a copy of the zoneinfo database, and a distribution containing the latest zoneinfo database will also be available at the Python Package Index, so it can be easily installed with the Python packaging tools such as ``easy_install`` or ``pip``. This could also be done on Unices that are no longer receiving updates and therefore have an outdated database. With such a mechanism Python would have full time zone support in the standard library on any platform, and a simple package installation would provide an updated time zone database on those platforms where the zoneinfo database isn't included, such as Windows, or on platforms where OS updates are no longer provided. The time zone support will be implemented by making the ``datetime`` module into a package, and adding time zone support to ``datetime`` based on Stuart Bishop's ``pytz`` module. Getting the local time zone --------------------------- On Unix there is no standard way of finding the name of the time zone that is being used. All the information that is available is the time zone abbreviations, such as ``EST`` and ``PDT``, but many of those abbreviations are ambiguous and therefore you can't rely on them to figure out which time zone you are located in. There is however a standard for finding the compiled time zone information since it's located in ``/etc/localtime``. Therefore it is possible to create a local time zone object with the correct time zone information even though you don't know the name of the time zone. A function in ``datetime`` should be provided to return the local time zone. The support for this will be made by integrating Lennart Regebro's ``tzlocal`` module into the new ``datetime`` module. For Windows it will look up the local Windows time zone name, and use a mapping between Windows time zone names and zoneinfo time zone names provided by the Unicode consortium to convert that to a zoneinfo time zone. The mapping should be updated before each major or bugfix release, scripts for doing so will be provided in the ``Tools/`` directory. Ambiguous times --------------- When changing over from daylight savings time (DST) the clock is turned back one hour. This means that the times during that hour happens twice, once with DST and then once without DST. Similarly, when changing to daylight savings time, one hour goes missing. The current time zone API can not differentiate between the two ambiguous times during a change from DST. For example, in Stockholm the time of 2012-11-28 02:00:00 happens twice, both at UTC 2012-11-28 00:00:00 and also at 2012-11-28 01:00:00. The current time zone API can not disambiguate this and therefore it's unclear which time should be returned:: # This could be either 00:00 or 01:00 UTC: >>> dt = datetime(2012, 10, 28, 2, 0, tzinfo=zoneinfo('Europe/Stockholm')) # But we can not specify which: >>> dt.astimezone(zoneinfo('UTC')) datetime.datetime(2012, 10, 28, 1, 0, tzinfo=<UTC>) ``pytz`` solved this problem by adding ``is_dst`` parameters to several methods of the tzinfo objects to make it possible to disambiguate times when this is desired. This PEP proposes to add these ``is_dst`` parameters to the relevant methods of the ``datetime`` API, and therefore add this functionality directly to ``datetime``. This is likely the hardest part of this PEP as this involves updating the C version of the ``datetime`` library with this functionality, as this involved writing new code, and not just reorganizing existing external libraries. Implementation API ================== The zoneinfo database --------------------- The latest version of the zoneinfo database should exist in the ``Lib/tzdata`` directory of the Python source control system. This copy of the database should be updated before every Python feature and bug-fix release, but not for releases of Python versions that are in security-fix-only-mode. Scripts to update the database will be provided in ``Tools/``, and the release instructions will be updated to include this update. New configure options ``--enable-internal-timezone-database`` and ``--disable-internal-timezone-database`` will be implemented to enable and disable the installation of this database when installing from source. A source install will default to installing them. Binary installers for systems that have a system-provided zoneinfo database may skip installing the included database since it would never be used for these platforms. For other platforms, for example Windows, binary installers must install the included database. Changes in the ``datetime``-module ---------------------------------- The public API of the new time zone support contains one new class, one new function, one new exception and four new collections. In addition to this, several methods on the datetime object gets a new ``is_dst`` parameter. New class ``dsttimezone`` ^^^^^^^^^^^^^^^^^^^^^^^^ This class provides a concrete implementation of the ``tzinfo`` base class that implements DST support. New function ``zoneinfo(name=None, db_path=None)`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This function takes a name string that must be a string specifying a valid zoneinfo time zone, i.e. "US/Eastern", "Europe/Warsaw" or "Etc/GMT". If not given, the local time zone will be looked up. If an invalid zone name is given, or the local time zone can not be retrieved, the function raises `UnknownTimeZoneError`. The function also takes an optional path to the location of the zoneinfo database which should be used. If not specified, the function will look for databases in the following order: 1. Check if the `tzdata-update` module is installed, and then use that database. 2. Use the database in ``/usr/share/zoneinfo``, if it exists. 3. Use the Python-provided database in ``Lib/tzdata``. If no database is found an ``UnknownTimeZoneError`` or subclass thereof will be raised with a message explaining that no zoneinfo database can be found, but that you can install one with the ``tzdata-update`` package. New parameter ``is_dst`` ^^^^^^^^^^^^^^^^^^^^^^^^ A new ``is_dst`` parameter is added to several methods to handle time ambiguity during DST changeovers. * ``tzinfo.utcoffset(dt, is_dst=False)`` * ``tzinfo.dst(dt, is_dst=False)`` * ``tzinfo.tzname(dt, is_dst=False)`` * ``datetime.astimezone(tz, is_dst=False)`` The ``is_dst`` parameter can be ``False`` (default), ``True``, or ``None``. ``False`` will specify that the given datetime should be interpreted as not happening during daylight savings time, i.e. that the time specified is after the change from DST. This is default to preserve existing behavior. ``True`` will specify that the given datetime should be interpreted as happening during daylight savings time, i.e. that the time specified is before the change from DST. ``None`` will raise an ``AmbiguousTimeError`` exception if the time specified was during a DST change over. It will also raise a ``NonExistentTimeError`` if a time is specified during the "missing time" in a change to DST. New exceptions ^^^^^^^^^^^^^^ * ``UnknownTimeZoneError`` This exception is a subclass of KeyError and raised when giving a time zone specification that can't be found:: >>> datetime.zoneinfo('Europe/New_York') Traceback (most recent call last): ... UnknownTimeZoneError: There is no time zone called 'Europe/New_York' * ``InvalidTimeError`` This exception serves as a base for ``AmbiguousTimeError`` and ``NonExistentTimeError``, to enable you to trap these two separately. It will subclass from ValueError, so that you can catch these errors together with inputs like the 29th of February 2011. * ``AmbiguousTimeError`` This exception is raised when giving a datetime specification that is ambiguous while setting ``is_dst`` to None:: >>> datetime(2012, 11, 28, 2, 0, tzinfo=zoneinfo('Europe/Stockholm'), is_dst=None) >>> Traceback (most recent call last): ... AmbiguousTimeError: 2012-10-28 02:00:00 is ambiguous in time zone Europe/Stockholm * ``NonExistentTimeError`` This exception is raised when giving a datetime specification for a time that due to daylight saving does not exist, while setting ``is_dst`` to None:: >>> datetime(2012, 3, 25, 2, 0, tzinfo=zoneinfo('Europe/Stockholm'), is_dst=None) >>> Traceback (most recent call last): ... NonExistentTimeError: 2012-03-25 02:00:00 does not exist in time zone Europe/Stockholm New collections ^^^^^^^^^^^^^^^ * ``all_timezones`` is the exhaustive list of the time zone names that can be used, listed alphabethically. * ``common_timezones`` is a list of useful, current time zones, listed alphabethically. The ``tzdata-update``-package ----------------------------- The zoneinfo database will be packaged for easy installation with ``easy_install``/``pip``/``buildout``. This package will not install any Python code, and will not contain any Python code except that which is needed for installation. It will be kept updated with the same tools as the internal database, but released whenever the ``zoneinfo``-database is updated, and use the same version schema. Differences from the ``pytz`` API ================================= * ``pytz`` has the functions ``localize()`` and ``normalize()`` to work around that ``tzinfo`` doesn't have is_dst. When ``is_dst`` is implemented directly in ``datetime.tzinfo`` they are no longer needed. * The ``timezone()`` function is called ``zoneinfo()`` to avoid clashing with the ``timezone`` class introduced in Python 3.2. * ``zoneinfo()`` will return the local time zone if called without arguments. * The class ``pytz.StaticTzInfo`` is there to provide the ``is_dst`` support for static time zones. When ``is_dst`` support is included in ``datetime.tzinfo`` it is no longer needed. * ``InvalidTimeError`` subclasses from ``ValueError``. Resources ========= * http://pytz.sourceforge.net/ * http://pypi.python.org/pypi/tzlocal * http://pypi.python.org/pypi/python-dateutil * http://unicode.org/cldr/data/common/supplemental/windowsZones.xml Copyright ========= This document has been placed in the public domain. [View Less]

1 0

Moderating the severity of the distutils freeze
by Nick Coghlan Feb. 7, 2013

Feb. 7, 2013

On Thu, Feb 7, 2013 at 6:45 AM, Éric Araujo <merwok(a)netwok.org> wrote: > Bug fixes are possible; changes to keep working with Python (i.e. pyc > files are generated in pycache directories after 3.2); changes to be > able to build on current OSes (e.g. following Mac dev tools location > change, introduction of Debian multiarch, etc.). > > Some bugs have been here for so long that everybody depends on them or > work around them, or they would be extremely painful to … [View More]

1 0

importlib hooray!
by Thomas Heller Feb. 6, 2013

Feb. 6, 2013

I have become a fan of the new python 3.3 importlib in the last few days. It has allowed me to write a ModuleMapper which I put into sys.metapath (in sitecustomize.py, for Python 3.3). This mapper currently does rename modules like 'Queue' or '_winreg' to the Python3 modules 'queue' or 'winreg' without the need to change my 2.7 sourcecode (changing the sourcecode is required when using six.moves to write 2.x/3.x compatible code). The six.moves approach has another problem with freeze tools (… [View More]

2 1