Mailman 3 July 2016 - Python-Dev

Re: [Python-Dev] [Python-checkins] devguide: Star self as idlelib expert. Mark other 2 as inactive.
by Martin Panter 12 Jul '16

12 Jul '16

On 12 July 2016 at 03:47, terry.reedy <python-checkins(a)python.org> wrote: > https://hg.python.org/devguide/rev/cc1c0dd798e7 Terry it looks like you accidentally added Christian back (undoing <https://hg.python.org/devguide/rev/cc1c0dd798e7>) > -xml.parsers.expat > -xml.sax > +xml.parsers.expat christian.heimes > +xml.sax christian.heimes > xml.sax.handler > xml.sax.saxutils > xml.sax.xmlreader > @@ -319,7 +319,7 @@ > bytecode benjamin.peterson, pitrou, georg.brandl, yselivanov > context managers ncoghlan > coverity scan christian.heimes, brett.cannon, twouters > -cryptography gregory.p.smith, dstufft > +cryptography christian.heimes, gregory.p.smith, dstufft > data formats mark.dickinson, georg.brandl > database lemburg > devguide ncoghlan, eric.araujo, ezio.melotti, willingc > @@ -341,7 +341,7 @@ > georg.brandl > str.format eric.smith > testing michael.foord, pitrou, ezio.melotti > -test coverage giampaolo.rodola > +test coverage giampaolo.rodola, christian.heimes > threads pitrou > time and dates lemburg, belopolsky > unicode lemburg, ezio.melotti, haypo, benjamin.peterson, pitrou

1 1

Python CFG Basic blocks
by Obiesie ike-nwosu 11 Jul '16

11 Jul '16

Hi, I am looking into how the python compiler generates basic blocks during the CFG generation process and my expectations from CFG theory seems to be at odds with how the python compiler actually generates its CFG. Take the following code snippet for example: def median(pool): copy = sorted(pool) size = len(copy) if size % 2 == 1: return copy[(size - 1) / 2] else: return (copy[size/2 - 1] + copy[size/2]) / 2 From my understanding of basic blocks in compilers, the above code snippet should have at least 3 basic blocks as follows: 1. Block 1 - all instructions up to and including those for the if test. 2. Block 2 - all instructions for the if body i.e the first return statement. 3. Block 3 - instructions for the else block i.e. the second return statement. My understanding of the the section on Control flow Graphs in the “Design of the CPython Compiler” also alludes to this - >> As an example, consider an ‘if’ statement with an ‘else’ block. The guard on the ‘if’ is a basic block which is pointed to by the basic block containing the code leading to the ‘if’ statement. The ‘if’ statement block contains jumps (which are exit points) to the true body of the ‘if’ and the ‘else’ body (which may be NULL), each of which are their own basic blocks. Both of those blocks in turn point to the basic block representing the code following the entire ‘if’ statement. The CPython compiler however seems to generate 2 basic blocks for the above snippets - 1. Block 1 - all instructions up to and including the if statement and the body of the if statement (the first return statement in this case) 2. Block 2 - instructions for the else block (the second return statement) Is there any reason for this or have I somehow missed something in the CFG generation process? Regards, Obi

2 1

Reminder: 3.6.0a3 shapshot 2016-07-11 12:00 UTC
by Ned Deily 08 Jul '16

08 Jul '16

The next alpha snapshot for the 3.6 release cycle is coming up in a couple of days. This is the third of four alphas we have planned. Keep that feature code coming in! As a reminder, alpha releases are intended to make it easier for the wider community to test the current state of new features and bug fixes for an upcoming Python release as a whole and for us to test the release process. During the alpha phase, features may be added, modified, or deleted up until the start of the beta phase. Alpha users beware! Looking ahead, the last alpha release, 3.6.0a4, will follow next month on 2016-08-15. This is a slight change from the previous schedule. Due to popular request, we're moved the feature code cutoff date, 3.6.0b1, from Wednesday, 09-07, to Monday, 09-12, to allow a bit more time after the end-of-summer holiday. In doing so, it seems to make sense to move the final alpha back a week to give a bit more time there for check-ins. To recap: 2016-07-11 ~12:00 UTC: code snapshot for 3.6.0 alpha 3 2016-08-15 ~12:00 UTC: code snapshot for 3.6.0 alpha 4. (**CHANGED**, was 08-08) now to 2016-09-12: Alpha phase (unrestricted feature development) 2016-09-12: 3.6.0 feature code freeze, 3.7.0 feature development begins (**CHANGED**, was 09-07) 2016-09-12 to 2016-12-04: 3.6.0 beta phase (bug and regression fixes, no new features) 2016-12-04 3.6.0 release candidate 1 (3.6.0 code freeze) 2016-12-16 3.6.0 release (3.6.0rc1 plus, if necessary, any dire emergency fixes) --Ned https://www.python.org/dev/peps/pep-0494/ -- Ned Deily nad(a)python.org -- []

1 0

Summary of Python tracker Issues
by Python tracker 08 Jul '16

08 Jul '16

ACTIVITY SUMMARY (2016-07-01 - 2016-07-08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5550 ( +5) closed 33676 (+28) total 39226 (+33) Open issues with patches: 2427 Issues opened (22) ================== #23908: Check path arguments of os functions for null character http://bugs.python.org/issue23908 reopened by koobs #26765: Factor out common bytes and bytearray implementation http://bugs.python.org/issue26765 reopened by serhiy.storchaka #27441: redundant assignments to ob_size of new ints that _PyLong_New http://bugs.python.org/issue27441 opened by Oren Milman #27442: expose the Android API level in sysconfig.get_config_vars() http://bugs.python.org/issue27442 opened by xdegaye #27444: Python doesn't build due to test_float.py broken on non-IEEE m http://bugs.python.org/issue27444 opened by stark #27445: Charset instance not passed to set_payload() http://bugs.python.org/issue27445 opened by claudep #27446: struct: allow per-item byte order http://bugs.python.org/issue27446 opened by zwol #27447: python -m doctest script_file_with_no_py_extension produces co http://bugs.python.org/issue27447 opened by towb #27448: Race condition in subprocess.Popen which causes a huge memory http://bugs.python.org/issue27448 opened by aonishuk #27450: bz2: BZ2File should expose compression level as an attribute http://bugs.python.org/issue27450 opened by joshtriplett #27451: gzip.py: Please save more of the gzip header for later examina http://bugs.python.org/issue27451 opened by joshtriplett #27452: IDLE: Cleanup config code http://bugs.python.org/issue27452 opened by terry.reedy #27453: $CPP invocation in configure must use $CPPFLAGS http://bugs.python.org/issue27453 opened by Chi Hsuan Yen #27454: PyUnicode_InternInPlace can use PyDict_SetDefault http://bugs.python.org/issue27454 opened by methane #27455: Fix tkinter examples to be PEP8 compliant http://bugs.python.org/issue27455 opened by John Hagen #27456: TCP_NODELAY http://bugs.python.org/issue27456 opened by j1m #27461: Optimize PNGs http://bugs.python.org/issue27461 opened by scop #27464: Document that SplitResult & friends are namedtuples http://bugs.python.org/issue27464 opened by kmike #27465: IDLE:Make help source menu entries unique and sorted. http://bugs.python.org/issue27465 opened by terry.reedy #27466: [Copy from github user macartur] time2netscape missing comma http://bugs.python.org/issue27466 opened by Robby Daigle #27468: Erroneous memory behaviour for objects created in another thre http://bugs.python.org/issue27468 opened by Adria Garriga #27469: Unicode filename gets crippled on Windows when drag and drop http://bugs.python.org/issue27469 opened by Drekin Most recent 15 issues with no replies (15) ========================================== #27469: Unicode filename gets crippled on Windows when drag and drop http://bugs.python.org/issue27469 #27468: Erroneous memory behaviour for objects created in another thre http://bugs.python.org/issue27468 #27464: Document that SplitResult & friends are namedtuples http://bugs.python.org/issue27464 #27451: gzip.py: Please save more of the gzip header for later examina http://bugs.python.org/issue27451 #27446: struct: allow per-item byte order http://bugs.python.org/issue27446 #27445: Charset instance not passed to set_payload() http://bugs.python.org/issue27445 #27435: ctypes and AIX - also for 2.7.X (and later) http://bugs.python.org/issue27435 #27428: Document WindowsRegistryFinder inherits from MetaPathFinder http://bugs.python.org/issue27428 #27427: Add new math module tests http://bugs.python.org/issue27427 #27426: Encoding mismatch causes some tests to fail on Windows http://bugs.python.org/issue27426 #27420: Docs for os.link - say what happens if link already exists http://bugs.python.org/issue27420 #27411: Possible different behaviour of explicit and implicit __dict__ http://bugs.python.org/issue27411 #27409: List socket.SO_*, SCM_*, MSG_*, IPPROTO_* symbols http://bugs.python.org/issue27409 #27408: Document importlib.abc.ExecutionLoader implements get_data() http://bugs.python.org/issue27408 #27404: Misc/NEWS: add [Security] prefix to Python 3.5.2 changelog http://bugs.python.org/issue27404 Most recent 15 issues waiting for review (15) ============================================= #27466: [Copy from github user macartur] time2netscape missing comma http://bugs.python.org/issue27466 #27461: Optimize PNGs http://bugs.python.org/issue27461 #27455: Fix tkinter examples to be PEP8 compliant http://bugs.python.org/issue27455 #27454: PyUnicode_InternInPlace can use PyDict_SetDefault http://bugs.python.org/issue27454 #27453: $CPP invocation in configure must use $CPPFLAGS http://bugs.python.org/issue27453 #27452: IDLE: Cleanup config code http://bugs.python.org/issue27452 #27448: Race condition in subprocess.Popen which causes a huge memory http://bugs.python.org/issue27448 #27445: Charset instance not passed to set_payload() http://bugs.python.org/issue27445 #27442: expose the Android API level in sysconfig.get_config_vars() http://bugs.python.org/issue27442 #27441: redundant assignments to ob_size of new ints that _PyLong_New http://bugs.python.org/issue27441 #27429: xml.sax.saxutils.escape doesn't escape multiple characters saf http://bugs.python.org/issue27429 #27427: Add new math module tests http://bugs.python.org/issue27427 #27423: Failed assertions when running test.test_os on Windows http://bugs.python.org/issue27423 #27419: Bugs in PyImport_ImportModuleLevelObject http://bugs.python.org/issue27419 #27413: Add an option to json.tool to bypass non-ASCII characters. http://bugs.python.org/issue27413 Top 10 most discussed issues (10) ================================= #27442: expose the Android API level in sysconfig.get_config_vars() http://bugs.python.org/issue27442 25 msgs #27456: TCP_NODELAY http://bugs.python.org/issue27456 9 msgs #27455: Fix tkinter examples to be PEP8 compliant http://bugs.python.org/issue27455 8 msgs #23908: Check path arguments of os functions for null character http://bugs.python.org/issue23908 7 msgs #27436: Strange code in selectors.KqueueSelector http://bugs.python.org/issue27436 7 msgs #27444: Python doesn't build due to test_float.py broken on non-IEEE m http://bugs.python.org/issue27444 7 msgs #27452: IDLE: Cleanup config code http://bugs.python.org/issue27452 7 msgs #27448: Race condition in subprocess.Popen which causes a huge memory http://bugs.python.org/issue27448 6 msgs #26765: Factor out common bytes and bytearray implementation http://bugs.python.org/issue26765 5 msgs #27081: Cannot capture sys.stderr output from an uncaught exception in http://bugs.python.org/issue27081 5 msgs Issues closed (28) ================== #11027: Implement sectionxform in configparser http://bugs.python.org/issue11027 closed by berker.peksag #19527: Test failures with COUNT_ALLOCS http://bugs.python.org/issue19527 closed by serhiy.storchaka #22198: Odd floor-division corner case http://bugs.python.org/issue22198 closed by serhiy.storchaka #22624: Bogus usage of floatclock in timemodule http://bugs.python.org/issue22624 closed by haypo #23034: Dynamically control debugging output http://bugs.python.org/issue23034 closed by serhiy.storchaka #24557: Refactor LibreSSL / EGD detection http://bugs.python.org/issue24557 closed by python-dev #27019: Reduce marshal stack depth for 2.7 on Windows debug build http://bugs.python.org/issue27019 closed by python-dev #27220: Add a pure Python version of 'collections.defaultdict' http://bugs.python.org/issue27220 closed by rhettinger #27248: Possible refleaks in PyType_Ready in error condition http://bugs.python.org/issue27248 closed by python-dev #27254: UAF in Tkinter module http://bugs.python.org/issue27254 closed by Emin Ghuliev #27332: Clinic: first parameter for module-level functions should be P http://bugs.python.org/issue27332 closed by serhiy.storchaka #27380: IDLE: add base Query dialog with ttk widgets http://bugs.python.org/issue27380 closed by terry.reedy #27410: DLL hijacking vulnerability in Python 3.5.2 installer http://bugs.python.org/issue27410 closed by steve.dower #27422: Deadlock when mixing threading and multiprocessing http://bugs.python.org/issue27422 closed by davin #27434: cross-building python 3.6 with an older interpreter fails http://bugs.python.org/issue27434 closed by xdegaye #27437: IDLE tests must be able to set user configuration values. http://bugs.python.org/issue27437 closed by terry.reedy #27438: Refactor simple iterators implementation http://bugs.python.org/issue27438 closed by serhiy.storchaka #27439: Add a product() function to the standard library http://bugs.python.org/issue27439 closed by rhettinger #27440: Trigonometric bug http://bugs.python.org/issue27440 closed by tim.peters #27443: __length_hint__() of bytearray iterator can return negative in http://bugs.python.org/issue27443 closed by serhiy.storchaka #27449: pip install --upgrade pip (Windows) http://bugs.python.org/issue27449 closed by zach.ware #27457: shlex.quote incorrectly quotes ampersants, pipes http://bugs.python.org/issue27457 closed by r.david.murray #27458: Allow subtypes of unicode/str to hit the optimized unicode_con http://bugs.python.org/issue27458 closed by haypo #27459: unintended changes occur when dealing with list of list http://bugs.python.org/issue27459 closed by eryksun #27460: Change bytes exception when overflow http://bugs.python.org/issue27460 closed by serhiy.storchaka #27462: NULL Pointer deref in binary_iop1 function http://bugs.python.org/issue27462 closed by r.david.murray #27463: Floor division is not the same as the floor of division http://bugs.python.org/issue27463 closed by r.david.murray #27467: distutils.config API different between <=3.5.1 and 3.5.2 http://bugs.python.org/issue27467 closed by berker.peksag

1 0

digital panels processing text sowftware write with Python language
by Hadi Montazami safavi 08 Jul '16

08 Jul '16

Dear SirsWith thanks, iam looking digital panels processing text sowftware write with Python languagefor my project.Could you please guide me if there is or guiding to write it personally.Thanking you in advance and awaiting.With best regards.M.Safavi

2 1

Request for CPython 3.5.3 release
by Yury Selivanov 08 Jul '16

08 Jul '16

Long story short, I've discovered that asyncio is broken in 3.5.2. Specifically, there is a callbacks race in `loop.sock_connect` which can make subsequent `loop.sock_sendall` calls to hang forever. This thing is very tricky and hard to detect and debug; I had to spend a few hours investigating what's going on with a failing unittest in uvloop (asyncio-compatible event loop). I can only imagine how hard it would be to understand what's going on in a larger codebase. For those who is interested, here's a PR for asyncio repo: https://github.com/python/asyncio/pull/366 It explains the bug in detail and there has a proposed patch to fix the problem. Larry and the release team: would it be possible to make an "emergency" 3.5.3 release? Going forward, we need to increase the number of functional tests for asyncio, as most of the current tests use mocks. I'm going to port all functional tests from uvloop to asyncio as a start. Yury

23 67

Re: [Python-Dev] Breaking up the stdlib (Was: release cadence)
by Paul Moore 06 Jul '16

06 Jul '16

On 6 July 2016 at 14:55, Steve Dower <steve.dower(a)python.org> wrote: > I think the wsgiref issue was that it wasn't in site-packages and so > couldn't be removed or upgraded. Having the dist-info available and putting > them in site-packages (or a new directory?) lets us handle > querying/replacing/removing using the existing tools we use for > distribution. That's the one. Thanks for the reminder. So either we move the stdlib (or parts of it) into site-packages, or pip needs to learn to handle a versioned stdlib. Cool. > Also, I think the version numbers do need to be independent of Python > version in case nothing changes between releases. If you develop something > using statistics==3.7, but statistics==3.6 is identical, how do you know you > can put the lower constraint? Alternatively, if it's statistics==1.0 in > both, people won't assume it has anything to do with the Python version. This boils down to whether we want to present the stdlib as a unified object tied to the Python release, or a set of modules no different from those on PyPI, that happen to be shipped with Python. I prefer the former view (it matches better how I think of "batteries included") whereas it looks like you prefer the latter (but don't see that as being in conflict with "batteries included"). Debating this in the abstract is probably not productive, so let's wait for a concrete PEP to thrash out details like this. Paul

2 2

Re: [Python-Dev] [Webmaster] Unsafe listing by Norton's "File Insight"
by Steve Holden 05 Jul '16

05 Jul '16

Hi Peter, While the humble webmasters can do little about this it's possible the developers can, so I am forwarding your email to their mailing list. regards Steve Steve Holden On Tue, Jul 5, 2016 at 3:30 AM, Peter via Webmaster <webmaster(a)python.org> wrote: > Hi > I'm a heavy user of Python on Windows, am a Basic PSF member and have > contributed to core Python. > The Python 2.7.12 Windows installer download is being marked as untrusted > by Norton Internet Security. I've been on chat with Symantec, and they've > said that I can't do anything about that rating, but that the site owner > can. > I've been pointed to: > https://safeweb.norton.com/help/site_owners > Interestingly, the 3.5.2 download is flagged as safe. > Hoping to get more Python out to users! > Thanks > Peter > > _______________________________________________ > Webmaster mailing list > Webmaster(a)python.org > https://mail.python.org/mailman/listinfo/webmaster >

3 5

PEP487: Simpler customization of class creation
by Martin Teichmann 05 Jul '16

05 Jul '16

Hi list, so this is the next round for PEP 487. During the last round, most of the comments were in the direction that a two step approach for integrating into Python, first in pure Python, later in C, was not a great idea and everything should be in C directly. So I implemented it in C, put it onto the issue tracker here: http://bugs.python.org/issue27366, and also modified the PEP accordingly. For those who had not been in the discussion, PEP 487 proposes to add two hooks, __init_subclass__ which is a classmethod called whenever a class is subclassed, and __set_owner__, a hook in descriptors which gets called once the class the descriptor is part of is created. While implementing PEP 487 I realized that there is and oddity in the type base class: type.__init__ forbids to use keyword arguments, even for the usual three arguments it has (name, base and dict), while type.__new__ allows for keyword arguments. As I plan to forward any keyword arguments to the new __init_subclass__, I stumbled over that. As I write in the PEP, I think it would be a good idea to forbid using keyword arguments for type.__new__ as well. But if people think this would be to big of a change, it would be possible to do it differently. Hoping for good comments Greetings Martin The PEP follows: PEP: 487 Title: Simpler customisation of class creation Version: $Revision$ Last-Modified: $Date$ Author: Martin Teichmann <lkb.teichmann(a)gmail.com>, Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 27-Feb-2015 Python-Version: 3.6 Post-History: 27-Feb-2015, 5-Feb-2016, 24-Jun-2016, 2-Jul-2016 Replaces: 422 Abstract ======== Currently, customising class creation requires the use of a custom metaclass. This custom metaclass then persists for the entire lifecycle of the class, creating the potential for spurious metaclass conflicts. This PEP proposes to instead support a wide range of customisation scenarios through a new ``__init_subclass__`` hook in the class body, and a hook to initialize attributes. The new mechanism should be easier to understand and use than implementing a custom metaclass, and thus should provide a gentler introduction to the full power Python's metaclass machinery. Background ========== Metaclasses are a powerful tool to customize class creation. They have, however, the problem that there is no automatic way to combine metaclasses. If one wants to use two metaclasses for a class, a new metaclass combining those two needs to be created, typically manually. This need often occurs as a surprise to a user: inheriting from two base classes coming from two different libraries suddenly raises the necessity to manually create a combined metaclass, where typically one is not interested in those details about the libraries at all. This becomes even worse if one library starts to make use of a metaclass which it has not done before. While the library itself continues to work perfectly, suddenly every code combining those classes with classes from another library fails. Proposal ======== While there are many possible ways to use a metaclass, the vast majority of use cases falls into just three categories: some initialization code running after class creation, the initalization of descriptors and keeping the order in which class attributes were defined. The first two categories can easily be achieved by having simple hooks into the class creation: 1. An ``__init_subclass__`` hook that initializes all subclasses of a given class. 2. upon class creation, a ``__set_owner__`` hook is called on all the attribute (descriptors) defined in the class, and The third category is the topic of another PEP 520. As an example, the first use case looks as follows:: >>> class SpamBase: ... # this is implicitly a @classmethod ... def __init_subclass__(cls, **kwargs): ... cls.class_args = kwargs ... super().__init_subclass__(cls, **kwargs) >>> class Spam(SpamBase, a=1, b="b"): ... pass >>> Spam.class_args {'a': 1, 'b': 'b'} The base class ``object`` contains an empty ``__init_subclass__`` method which serves as an endpoint for cooperative multiple inheritance. Note that this method has no keyword arguments, meaning that all methods which are more specialized have to process all keyword arguments. This general proposal is not a new idea (it was first suggested for inclusion in the language definition `more than 10 years ago`_, and a similar mechanism has long been supported by `Zope's ExtensionClass`_), but the situation has changed sufficiently in recent years that the idea is worth reconsidering for inclusion. The second part of the proposal adds an ``__set_owner__`` initializer for class attributes, especially if they are descriptors. Descriptors are defined in the body of a class, but they do not know anything about that class, they do not even know the name they are accessed with. They do get to know their owner once ``__get__`` is called, but still they do not know their name. This is unfortunate, for example they cannot put their associated value into their object's ``__dict__`` under their name, since they do not know that name. This problem has been solved many times, and is one of the most important reasons to have a metaclass in a library. While it would be easy to implement such a mechanism using the first part of the proposal, it makes sense to have one solution for this problem for everyone. To give an example of its usage, imagine a descriptor representing weak referenced values:: import weakref class WeakAttribute: def __get__(self, instance, owner): return instance.__dict__[self.name] def __set__(self, instance, value): instance.__dict__[self.name] = weakref.ref(value) # this is the new initializer: def __set_owner__(self, owner, name): self.name = name While this example looks very trivial, it should be noted that until now such an attribute cannot be defined without the use of a metaclass. And given that such a metaclass can make life very hard, this kind of attribute does not exist yet. Initializing descriptors could simply be done in the ``__init_subclass__`` hook. But this would mean that descriptors can only be used in classes that have the proper hook, the generic version like in the example would not work generally. One could also call ``__set_owner__`` from whithin the base implementation of ``object.__init_subclass__``. But given that it is a common mistake to forget to call ``super()``, it would happen too often that suddenly descriptors are not initialized. Key Benefits ============ Easier inheritance of definition time behaviour ----------------------------------------------- Understanding Python's metaclasses requires a deep understanding of the type system and the class construction process. This is legitimately seen as challenging, due to the need to keep multiple moving parts (the code, the metaclass hint, the actual metaclass, the class object, instances of the class object) clearly distinct in your mind. Even when you know the rules, it's still easy to make a mistake if you're not being extremely careful. Understanding the proposed implicit class initialization hook only requires ordinary method inheritance, which isn't quite as daunting a task. The new hook provides a more gradual path towards understanding all of the phases involved in the class definition process. Reduced chance of metaclass conflicts ------------------------------------- One of the big issues that makes library authors reluctant to use metaclasses (even when they would be appropriate) is the risk of metaclass conflicts. These occur whenever two unrelated metaclasses are used by the desired parents of a class definition. This risk also makes it very difficult to *add* a metaclass to a class that has previously been published without one. By contrast, adding an ``__init_subclass__`` method to an existing type poses a similar level of risk to adding an ``__init__`` method: technically, there is a risk of breaking poorly implemented subclasses, but when that occurs, it is recognised as a bug in the subclass rather than the library author breaching backwards compatibility guarantees. New Ways of Using Classes ========================= This proposal has many usecases like the following. In the examples, we still inherit from the ``SubclassInit`` base class. This would become unnecessary once this PEP is included in Python directly. Subclass registration --------------------- Especially when writing a plugin system, one likes to register new subclasses of a plugin baseclass. This can be done as follows:: class PluginBase(Object): subclasses = [] def __init_subclass__(cls, **kwargs): super().__init_subclass__(**kwargs) cls.subclasses.append(cls) In this example, ``PluginBase.subclasses`` will contain a plain list of all subclasses in the entire inheritance tree. One should note that this also works nicely as a mixin class. Trait descriptors ----------------- There are many designs of Python descriptors in the wild which, for example, check boundaries of values. Often those "traits" need some support of a metaclass to work. This is how this would look like with this PEP:: class Trait: def __get__(self, instance, owner): return instance.__dict__[self.key] def __set__(self, instance, value): instance.__dict__[self.key] = value def __set_owner__(self, owner, name): self.key = name Implementation Details ====================== For those who prefer reading Python over english, the following is a Python equivalent of the C API changes proposed in this PEP, where the new ``object`` and ``type`` defined here inherit from the usual ones:: import types class type(type): def __new__(cls, *args, **kwargs): if len(args) == 1: return super().__new__(cls, args[0]) name, bases, ns = args init = ns.get('__init_subclass__') if isinstance(init, types.FunctionType): ns['__init_subclass__'] = classmethod(init) self = super().__new__(cls, name, bases, ns) for k, v in self.__dict__.items(): func = getattr(v, '__set_owner__', None) if func is not None: func(self, k) super(self, self).__init_subclass__(**kwargs) return self def __init__(self, name, bases, ns, **kwargs): super().__init__(name, bases, ns) class object: @classmethod def __init_subclass__(cls): pass class object(object, metaclass=type): pass In this code, first the ``__set_owner__`` are called on the descriptors, and then the ``__init_subclass__``. This means that subclass initializers already see the fully initialized descriptors. This way, ``__init_subclass__`` users can fix all descriptors again if this is needed. Another option would have been to call ``__set_owner__`` in the base implementation of ``object.__init_subclass__``. This way it would be possible event to prevent ``__set_owner__`` from being called. Most of the times, however, such a prevention would be accidental, as it often happens that a call to ``super()`` is forgotten. Another small change should be noted here: in the current implementation of CPython, ``type.__init__`` explicitly forbids the use of keyword arguments, while ``type.__new__`` allows for its attributes to be shipped as keyword arguments. This is weirdly incoherent, and thus the above code forbids that. While it would be possible to retain the current behavior, it would be better if this was fixed, as it is probably not used at all: the only use case would be that at metaclass calls its ``super().__new__`` with *name*, *bases* and *dict* (yes, *dict*, not *namespace* or *ns* as mostly used with modern metaclasses) as keyword arguments. This should not be done. As a second change, the new ``type.__init__`` just ignores keyword arguments. Currently, it insists that no keyword arguments are given. This leads to a (wanted) error if one gives keyword arguments to a class declaration if the metaclass does not process them. Metaclass authors that do want to accept keyword arguments must filter them out by overriding ``__init___``. In the new code, it is not ``__init__`` that complains about keyword arguments, but ``__init_subclass__``, whose default implementation takes no arguments. In a classical inheritance scheme using the method resolution order, each ``__init_subclass__`` may take out it's keyword arguments until none are left, which is checked by the default implementation of ``__init_subclass__``. Rejected Design Options ======================= Calling the hook on the class itself ------------------------------------ Adding an ``__autodecorate__`` hook that would be called on the class itself was the proposed idea of PEP 422. Most examples work the same way or even better if the hook is called on the subclass. In general, it is much easier to explicitly call the hook on the class in which it is defined (to opt-in to such a behavior) than to opt-out, meaning that one does not want the hook to be called on the class it is defined in. This becomes most evident if the class in question is designed as a mixin: it is very unlikely that the code of the mixin is to be executed for the mixin class itself, as it is not supposed to be a complete class on its own. The original proposal also made major changes in the class initialization process, rendering it impossible to back-port the proposal to older Python versions. More importantly, having a pure Python implementation allows us to take two preliminary steps before before we actually change the interpreter, giving us the chance to iron out all possible wrinkles in the API. Other variants of calling the hook ---------------------------------- Other names for the hook were presented, namely ``__decorate__`` or ``__autodecorate__``. This proposal opts for ``__init_subclass__`` as it is very close to the ``__init__`` method, just for the subclass, while it is not very close to decorators, as it does not return the class. Requiring an explicit decorator on ``__init_subclass__`` -------------------------------------------------------- One could require the explicit use of ``@classmethod`` on the ``__init_subclass__`` decorator. It was made implicit since there's no sensible interpretation for leaving it out, and that case would need to be detected anyway in order to give a useful error message. This decision was reinforced after noticing that the user experience of defining ``__prepare__`` and forgetting the ``@classmethod`` method decorator is singularly incomprehensible (particularly since PEP 3115 documents it as an ordinary method, and the current documentation doesn't explicitly say anything one way or the other). A more ``__new__``-like hook ---------------------------- In PEP 422 the hook worked more like the ``__new__`` method than the ``__init__`` method, meaning that it returned a class instead of modifying one. This allows a bit more flexibility, but at the cost of much harder implementation and undesired side effects. Adding a class attribute with the attribute order ------------------------------------------------- This got its own PEP 520. History ======= This used to be a competing proposal to PEP 422 by Nick Coghlan and Daniel Urban. PEP 422 intended to achieve the same goals as this PEP, but with a different way of implementation. In the meantime, PEP 422 has been withdrawn favouring this approach. References ========== .. _more than 10 years ago: http://mail.python.org/pipermail/python-dev/2001-November/018651.html .. _Zope's ExtensionClass: http://docs.zope.org/zope_secrets/extensionclass.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

3 4

Automating the maintenance release pipeline (was Re: Request for CPython 3.5.3 release)
by Nick Coghlan 04 Jul '16

04 Jul '16

On 4 July 2016 at 00:39, Guido van Rossum <guido(a)python.org> wrote: > Another thought recently occurred to me. Do releases really have to be > such big productions? A recent ACM article by Tom Limoncelli[1] > reminded me that we're doing releases the old-fashioned way -- > infrequently, and with lots of manual labor. Maybe we could > (eventually) try to strive for a lighter-weight, more automated > release process? It would be less work, and it would reduce stress for > authors of stdlib modules and packages -- there's always the next > release. I would think this wouldn't obviate the need for carefully > planned and timed "big deal" feature releases, but it could make the > bug fix releases *less* of a deal, for everyone. Yes, getting the maintenance releases to the point of being largely automated would be beneficial. However, I don't think the problem is lack of desire for that outcome, it's that maintaining the release toolchain pretty much becomes a job at that point, as you really want to be producing nightly builds (since the creation of those nightlies in effect becomes the regression test suite for the release toolchain), and you also need to more strictly guard against even temporary regressions in the maintenance branches. There are some variants we could pursue around that model (e.g. automating Python-only updates without automating updates that require rebuilding the core interpreter binaries for Windows and Mac OS X), but none of it is the kind of thing likely to make anyone say "I want to work on improving this in my free time". Even for commercial redistributors, it isn't easy for us to make the business case for assigning someone to work on it, since we're generally working from the source trees rather than the upstream binary releases. I do think it's worth putting this into our bucket of "ongoing activities we could potentially propose to the PSF for funding", though. I know Ewa (Jodlowska, the PSF's Director of Operations) is interested in better supporting the Python development community directly (hence https://donate.pypi.io/ ) in addition to the more indirect community building efforts like PyCon US and the grants program, so I've been trying to build up a mental list of CPython development pain points where funded activities could potentially improve the contributor experience for volunteers. So far I have: - issue triage (including better acknowledging folks that help out with triage efforts) - patch review (currently "wait and see" pending the impact of the GitHub migration) - nightly pre-release builds (for ease of contribution without first becoming a de facto C developer and to help make life easier for release managers) That last one is a new addition to my list based on this thread, and I think it's particularly interesting in that it would involve a much smaller set of target users than the first two (with the primary stakeholders being the release managers and the folks preparing the binary installers), but also a far more concrete set of deliverables (i.e. nightly binary builds being available for active development and maintenance branches for at least Windows and Mac OS X, and potentially for the manylinux1 baseline API defined in PEP 513) Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

3 3

Python-Dev July 2016