Mailman 3 December 2017 - Python-ideas

f-string literals by default?
by Neil Schemenauer Dec. 5, 2017

Dec. 5, 2017

I think most people who have tried f-strings have found them handy. Could we transition to making default string literal into an f-string? I think there is a smooth migration path. f-strings without embedded expressions already compile to the same bytecode as normal string literals. I.e. no overhead. The issue will be literal strings that contain the f-string format characters. We could add a future import, e.g. from __future__ import fstring_literals that would make all literal … [View More]

5 6

Data Classes in Kotlin
by Juancarlo Añez Dec. 5, 2017

Dec. 5, 2017

I thought this might be interesting input for the discussions about "data classes" in Python: https://kotlinlang.org/docs/reference/data-classes.html I think that the use of "data class" as syntax is kind of cool, but what really matters is the semantics they chose for Kotlin. -- Juancarlo *Añez*

1 0

PEP 447: Adding type.__getdescriptor__
by Ronald Oussoren Dec. 3, 2017

Dec. 3, 2017

Hi, I’m trying to get up to speed again w.r.t. PEP 447 (and to be honest, with CPython’s development workflow as its been too long since I’ve actually done work on the CPython codebase…). Let me start by recapping the PEP (<https://www.python.org/dev/peps/pep-0447/ <https://www.python.org/dev/peps/pep-0447/>>) is about: The problem I’m trying to solve is that super.__getattribute__ basicy assumes looking at the __dict__ of classes on the MRO is all that’s needed to check if those … [View More]classes provide an attribute. The core of both object.__getattribute__ and super.__getattribute__ for finding a descriptor on the class is basically (from the PEP): def _PyType_Lookup(tp, name): mro = tp.mro() assert isinstance(mro, tuple) for base in mro: assert isinstance(base, type) try: return base.__dict__[name] except KeyError: pass return None Note that the real implementation in in C, and accesses base.__dict__ directly instead of through Python’s attribute lookup (which would lead to infinite recursion). This is problematic when the class is dynamically populated, as the descriptor may not yet be present in the class __dict__. This is not a problem for normal attribute lookup, as you can replace the __getattribute__ method of the class to do the additional work (at the cost of having to reimplement all of __getattribute__). This is a problem for super() calls though, as super will unconditionally use the lookup code above. The PEP proposed to replace “base.__dict__[name]” by “base.__class__.__getdescriptor__(name)” (once again, in C with direct access to the two slots instead of accessing them through normal attribute lookup). I have two open questions about this PEP: 1) Last time around Mark Shannon worried that this introduces infinite recursion in the language itself (in my crummy summary, please read this message to get the real concern <https://mail.python.org/pipermail/python-dev/2015-July/140938.html <https://mail.python.org/pipermail/python-dev/2015-July/140938.html>>). Is this truly a problem? I don’t think there is a problem, but I’m worried that I don’t fully understand Mark’s concerns. 2) PEP 487 introduced __init_subclass__ as a class method to avoid having to write a metaclass for a number of use cases. My PEP currently does require a metaclass, but it might be nicer to switch to a regular class method instead (like __init_subclass__). My primary usecase for this PEP is PyObjC, which does populate the class dictionary on demand for performance and correctness reasons and PyObC therefore currently includes a replacement for super. For PyObjC’s implementation making __getdescriptor__ a classmethod would be a win as this would avoid creating yet another level of metaclasses in C code (PyObjC already has a Python class and metaclass for every Objective-C class, PEP 447 would require the introduction of a metaclass for those metaclasses). BTW. Two other open issues w.r.t. this PEP are forward porting the patch (in issue 18181) and performing benchmarks. The patch doesn’t apply cleanly to the current tip of the tree, I’ll try to update it tomorrow. And with some luck will manage to update PyObjC’s implementation as well to validate the design. Ronald [View Less]

2 9

Provide a way to import module without exec body
by Neil Schemenauer Dec. 3, 2017

Dec. 3, 2017

I have been working on reducing Python statup time. It would be nice if there was some way to load a module into memory without exec of its body code. I'm sure other people have wished for this. Perhaps there could be a new special function, similar to __import__ for this purpose. E.g. __load_module__(). To actually execute the module, I had the idea to make module objects callable, i.e. tp_call for PyModule_Type. That's a little too cute though and will cause confusion. Maybe instead, … [View More]

7 10

Re: [Python-ideas] Provide a way to import module without exec body
by Brett Cannon Dec. 3, 2017

Dec. 3, 2017

On Fri, 1 Dec 2017 at 10:11 Neil Schemenauer <neil(a)python.ca> wrote: > On 2017-12-01, Chris Angelico wrote: > > Can you elaborate on where this is useful, please? > > Introspection tools, for example, might want to look at the module > without executing it. Also, it is a building block to make lazy loading > of modules work. As Nick points out, importlib can do this already. > > Currently, the IMPORT_NAME both loads the code for a module and also > … [View More]executes it. The exec happens fairly deep in the guts of importlib. > This makes import.c and ceval.c mutually recursive. The locking gets > complicated. There are hacks like _call_with_frames_removed() to hide > the recursion going on. > > Instead, we could have two separate opcodes, one that gets the module > but does not exec it (i.e. a function like __import__() that returns a > future) and another opcode that actually does the execution. Figuring > out all the details is complicated. > > Possible benefits: > > - importlib is simpler > > - reduce the amount of stack space used (removing recursion by > "continuation passing style"). > > - makes profiling Python easier. Tools like valgrind get confused > by call cycle between ceval.c and import.c. > > - easier to implement lazy loading of modules (not necessarily a > standard Python feature but will make 3rd party implementations > cleaner) > > I'm CCing Brett as I'm sure he has thoughts on this, given his intimate > knowledge of importlib. To me, it seems like __import__() has a > terribly complicated API because it does so many different things. > I have always assumed the call signature for __import__() was because the import-related opcodes pushed so much logic into the function instead of doing it in opcodes (I actually blogged about this at https://snarky.ca/if-i-were-designing-imort-from-scratch/). Heck, the thing takes in locals() and yet never uses them (and its use of globals() is restricted to specific values so it really doesn't need to be quite so broad). Basically I wished __import__() looked like importlib.import_module(). > > Maybe two opcodes is not even enough. Maybe we should have one to > resolve relative imports (i.e. import.c:resolve_name), one to load but > not exec a module given its absolute name (i.e. _find_and_load() > without the exec), one to exec a loaded module, one or more to handle > the horror of "fromlist" (i.e. _handle_fromlist()). > I have always wanted to at least break up getting the module and fromlist as separate opcodes, so +1 for that. Name resolution could potentially be done as an opcode as it relies on execution state pulled from the globals of the module, but the logic also isn't difficult so +0 for that (i.e. making an opcode that calls something more like importlib.import_module() is more critical to me than eliminating the 'package' argument to that call, but I don't view it as a bad thing to have another opcode for that either). As for the completely separating the loading and execution, I don't have a need for what's being proposed so I don't have an opinion. I basically made sure Eric Snow structured specs so that lazy loading as currently supported works so I got what I wanted for basic lazy importing (short of the PyPI package I keep talking about writing to add a nicer API around lazy importing :) . -Brett > > Regards, > > Neil > [View Less]

3 3

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?
by Steven D'Aprano Dec. 3, 2017

Dec. 3, 2017

On Tue, Nov 28, 2017 at 12:31:06PM -0800, Raymond Hettinger wrote: > > > I also cc python-dev to see if anybody here is strongly in favor or against this inclusion. > > Put me down for a strong -1. The proposal would occasionally save a > few keystokes but comes at the expense of giving Python a more Perlish > look and a more arcane feel. I think that's an unfair characterisation of the benefits of the PEP. It's not just "a few keystrokes". Ironically, the … [View More]

11 21

How assignment should work with generators?
by Kirill Balunov Dec. 2, 2017

Dec. 2, 2017

Currently during assignment, when target list is a comma-separated list of targets (*without "starred" target*) the rule is that the object (rhs) must be an iterable with the same number of items as there are targets in the target list. That is, no check is performed on the number of targets present, and if something goes wrong the ValueError is raised. To show this on simple example: >>> from itertools import count, islice >>> it = count() >>> x, y = it >>… [View More]

14 61

Add a dict with the attribute access capability
by Serhiy Storchaka Dec. 1, 2017

Dec. 1, 2017

In 3.7 I have removed an old-deprecated plistlib.Dict. [1] Actually it already was deprecated when the plistlib module was added to the regular stdlib in Python 2.6. This is a dict subclass which allows to access items as attributes. d = plistlib.Dict() d['a'] = 1 assert d.a == 1 d.b = 2 assert d['b'] == 2 Raymond noticed that that capability seemed nice to have. What do you think about reviving this type as general purpose type in collections or types? Perhaps it can be convenient for … [View More]

12 12

Re: [Python-ideas] How assignment should work with generators?
by Alon Snir Dec. 1, 2017

Dec. 1, 2017

On Wed, Nov 29, 2017 at 5:46 AM, Alon Snir <AlonSnir(a)hotmail.com> wrote: > I would like to mention that the issue of assignment to a target list, is also relevant to the case of elementwise assignment to a mutable sequence (e.g. lists and arrays). Here as well, the rhs of the assignment statement states the number of elements to be assigned. Consequently, the following example will get into infinite loop: > >>>> from itertools import count >>>> A = [] >&… [View More]gt;>> A[:] = count() > > Writing "A[:2] = count()" will cause the same result. > Here as well, it is currently safer to use islice if the rhs is a generator or an iterator. e.g.: > >>>> it = count() >>>> A[:] = islice(it,2) > > In my opinion, it is be better to devise a solution that could be applied in both cases. Maybe a new kind of assignment operator that will be dedicated to this kind of assignment. i.e. elementwise assignment with restriction on the number of elements to be assigned, based on the length of the lhs object (or the number of targets in the target list). > ChrisA wrote: Hmm. The trouble is that slice assignment doesn't have a fixed number of targets. If you say "x, y = spam", there's a clear indication that 'spam' needs to provide exactly two values; but "A[:] = spam" could have any number of values, and it'll expand or shrink the list accordingly. Rhodri James wrote: Flatly, no. It is better not to ask for things you don't want in the first place, in this case the infinite sequence. Still, don't let me discourage you from working on this. If you can define how such an assignment would work, or even the length of A[:] as an assignment target, I'm not going to dismiss it out of hand. My answer: The idea is to define an alternative assignment rule, that is to assign exactly as many elements as the current length of the lhs object (without expanding or shrinking it). Suppose "?=" is the operator for the alternative assignment rule; A=[None]*2; and "iterator" is any iterator (finite or infinite). In this case, the following code: >>> A ?= iterator would behave like this: >>> A[:] = islice(iterator, 2) # where 2 is the length of A And as suggested earlier for the case of assignment to a target list, the following code: >>> x, y ?= iterator would behave like this: >>> x, y = islice(iterator, 2) # where 2 is the number of targets Regarding the length issue: Is there any difficulty in finding the length of a sliced sequence? After all, the range object has a len method. Therefore, the length of A[s] (where s is a slice object) could be evaluated as follows: >>> len(range(*s.indices(len(A)))) [View Less]

7 10

Stub class for Generic to further improve PEP 560
by Ilya Kulakov Dec. 1, 2017

Dec. 1, 2017

As explained in PEP 560, creation of Generic subclasses is much slower (~7x on my machine). My guess is that a limited number of applications actually need Generic classes at "production" runtime (i.e. when not under analysis of mypy or IDE). I propose a special class that will alias typing.Generic whenever typing.TYPE_CHECKING == True and a no-op stub otherwise. Best Regards, Ilya Kulakov

2 7