It would be useful in many scenarios for values in collections.Counter to
be allowed to be floating point. I know that Counter nominally emulates a
multiset, which would suggest only integer values, but in a more general
sense, it could be an accumulator of either floating point or integer data.
As near as I can tell, Collection already does support float values in both
Python 2.7 and 3.6, and the way the code is implemented, this change should
be a no-op. All that is required is to update the documentation to say
floating-point values are allowed, as it currently says only integers are
Consider the following code, which creates a generator that
immediately returns 1, and then catches the StopIteration twice.
if False: yield
except StopIteration as e:
raise ValueError("Generator was not ready to stop.")
gen = generatorfunction()
get_return(gen) #=> 1
get_return(gen) #=> None
The first time StopIteration is raised, it contains the returned
value. If StopIteration is forced again, the value is missing.
What about keeping the return value for subsequent raises? Perhaps as
an attribute on the generator object? The main disadvantage is that
it'd add another reference, keeping the return value alive as long as
the generator is alive. However, I don't think you'll want to keep a
dead generator around anyway.
status quo: the return value is only available on the first stop. C#
does not have the concept of returning from an iterator.
Background: I made a trampoline for a toy problem, using generators as
coroutines to recurse.
When it came time to memoize it, I had to couple the memoization with
the trampoline, because I could not cache the answer before it was
computed, and I could not cache the generator object because it would
not remember its return value later.
I would have attached the returned value to the generator object, but
for some reason, generator and coroutine objects can't take
attributes. Maybe I should ask for that feature instead.
Either feature would allow the concept of a coroutine that is also a thunk.
Since PEP 557 "Data Classes" and PEP 526 "Syntax for Variable
Annotations" are accepted and become part of the language. Is it worth
writing a proposal about decorator-version for creating an enumeration?
from enum import enum
@enum(unique=True, int_=False, flag=False, ...):
RED : auto
BLUE : auto
Despite the fact that Functional API to create enums already exists, it
seems to me that decorator-version will allow to unify these two relatively
young residents of the standard library. In addition, the idea of "Not
having to specify values for enums", which at the time seemed to involve
much magic in the implementation, becomes part of the language. Of course,
PEP 526 unequivocally says that it does not allow one to annotate the types
of variables when tuple unpacking is used. But in any case, I find the
variant with the decorator to be an interersting idea.
With kind regards, -gdg
 PEP 557 "Data Classes" <https://www.python.org/dev/peps/pep-0557/>
 PEP 526 "Syntax for Variable Annotations"
 Not having to specify values for enums.
I can't believe this hasn't been brought up before, but searching the web,
and python-ideas, and all the PEPs has found nothing (could be my lame
google-fu), so here goes:
Recent python has moved toward a "key" function for customized sorting:
key is also used (according to
min(), max(), heapq.nlargest(), heapq.nsmallest(), itertools.groupby()
with this fairly broad use, it seems it's becoming a fairly universal
protocol for ordering.
However, if you are writing a custom class, and want to make it "sortable",
you need to define (some of) the total comparison operators, which
presumably are then called O(n logn) number of times for comparisons when
Or provide a sort key function when you actually do the sorting, which
requires some inside knowledge of the objects you are sorting.
But what if there was a sort key magic method:
__key__ or __sort_key__ (or whatever)
that would be called by the sorting functions if:
no key function was specified
It seems this would provide a easy way to make custom classes sortable that
would be nicer for end users (not writing key functions), and possibly more
performant in the "usual" case.
In fact, it's striking me that there may well be classes that are defining
the comparison magic methods not because they want the objects to "work"
with the comparison operators, but because that want them to work with sort
and min, and max, and...
hmm, perhaps a __key__ method could even be used by the comparison
operators, though that could result in pretty weird results when comparing
two different types.
So: has this already been brought up and rejected?
Am I imagining the performance benefits?
Is sorting-related functionally too special-case to deserve a protocol?
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
I think most people who have tried f-strings have found them handy.
Could we transition to making default string literal into an
f-string? I think there is a smooth migration path.
f-strings without embedded expressions already compile to the same
bytecode as normal string literals. I.e. no overhead. The issue
will be literal strings that contain the f-string format characters.
We could add a future import, e.g.
from __future__ import fstring_literals
that would make all literal strings in the module into f-strings.
In some future release, we could warn about literal strings in
modules without the future import that contain f-string format
characters. Eventually, we can change the default.
To make migration easier, we can provide a source-to-source
translation tool. It is quite simple to do that using
the tokenizer module.
I thought this might be interesting input for the discussions about "data
classes" in Python:
I think that the use of "data class" as syntax is kind of cool, but what
really matters is the semantics they chose for Kotlin.
I’m trying to get up to speed again w.r.t. PEP 447 (and to be honest, with CPython’s development workflow as its been too long since I’ve actually done work on the CPython codebase…).
Let me start by recapping the PEP (<https://www.python.org/dev/peps/pep-0447/ <https://www.python.org/dev/peps/pep-0447/>>) is about: The problem I’m trying to solve is that super.__getattribute__ basicy assumes looking at the __dict__ of classes on the MRO is all that’s needed to check if those classes provide an attribute. The core of both object.__getattribute__ and super.__getattribute__ for finding a descriptor on the class is basically (from the PEP):
def _PyType_Lookup(tp, name):
mro = tp.mro()
assert isinstance(mro, tuple)
for base in mro:
assert isinstance(base, type)
Note that the real implementation in in C, and accesses base.__dict__ directly instead of through Python’s attribute lookup (which would lead to infinite recursion).
This is problematic when the class is dynamically populated, as the descriptor may not yet be present in the class __dict__. This is not a problem for normal attribute lookup, as you can replace the __getattribute__ method of the class to do the additional work (at the cost of having to reimplement all of __getattribute__). This is a problem for super() calls though, as super will unconditionally use the lookup code above.
The PEP proposed to replace “base.__dict__[name]” by “base.__class__.__getdescriptor__(name)” (once again, in C with direct access to the two slots instead of accessing them through normal attribute lookup).
I have two open questions about this PEP:
1) Last time around Mark Shannon worried that this introduces infinite recursion in the language itself (in my crummy summary, please read this message to get the real concern <https://mail.python.org/pipermail/python-dev/2015-July/140938.html <https://mail.python.org/pipermail/python-dev/2015-July/140938.html>>). Is this truly a problem? I don’t think there is a problem, but I’m worried that I don’t fully understand Mark’s concerns.
2) PEP 487 introduced __init_subclass__ as a class method to avoid having to write a metaclass for a number of use cases. My PEP currently does require a metaclass, but it might be nicer to switch to a regular class method instead (like __init_subclass__).
My primary usecase for this PEP is PyObjC, which does populate the class dictionary on demand for performance and correctness reasons and PyObC therefore currently includes a replacement for super. For PyObjC’s implementation making __getdescriptor__ a classmethod would be a win as this would avoid creating yet another level of metaclasses in C code (PyObjC already has a Python class and metaclass for every Objective-C class, PEP 447 would require the introduction of a metaclass for those metaclasses).
BTW. Two other open issues w.r.t. this PEP are forward porting the patch (in issue 18181) and performing benchmarks. The patch doesn’t apply cleanly to the current tip of the tree, I’ll try to update it tomorrow. And with some luck will manage to update PyObjC’s implementation as well to validate the design.
I have been working on reducing Python statup time. It would be
nice if there was some way to load a module into memory without exec
of its body code. I'm sure other people have wished for this.
Perhaps there could be a new special function, similar to __import__
for this purpose. E.g. __load_module__(). To actually execute the
module, I had the idea to make module objects callable, i.e. tp_call
for PyModule_Type. That's a little too cute though and will cause
confusion. Maybe instead, add a function attribute to modules, e.g.
I have a little experimental code, just a small step:
We need importlib to give us the module object and the bytecode
without doing the exec(). My hackish solution is to set properties
on __spec__ and then have PyImport_ImportModuleLevelObject() do the
On Fri, 1 Dec 2017 at 10:11 Neil Schemenauer <neil(a)python.ca> wrote:
> On 2017-12-01, Chris Angelico wrote:
> > Can you elaborate on where this is useful, please?
> Introspection tools, for example, might want to look at the module
> without executing it. Also, it is a building block to make lazy loading
> of modules work. As Nick points out, importlib can do this already.
> Currently, the IMPORT_NAME both loads the code for a module and also
> executes it. The exec happens fairly deep in the guts of importlib.
> This makes import.c and ceval.c mutually recursive. The locking gets
> complicated. There are hacks like _call_with_frames_removed() to hide
> the recursion going on.
> Instead, we could have two separate opcodes, one that gets the module
> but does not exec it (i.e. a function like __import__() that returns a
> future) and another opcode that actually does the execution. Figuring
> out all the details is complicated.
> Possible benefits:
> - importlib is simpler
> - reduce the amount of stack space used (removing recursion by
> "continuation passing style").
> - makes profiling Python easier. Tools like valgrind get confused
> by call cycle between ceval.c and import.c.
> - easier to implement lazy loading of modules (not necessarily a
> standard Python feature but will make 3rd party implementations
> I'm CCing Brett as I'm sure he has thoughts on this, given his intimate
> knowledge of importlib. To me, it seems like __import__() has a
> terribly complicated API because it does so many different things.
I have always assumed the call signature for __import__() was because the
import-related opcodes pushed so much logic into the function instead of
doing it in opcodes (I actually blogged about this at
https://snarky.ca/if-i-were-designing-imort-from-scratch/). Heck, the thing
takes in locals() and yet never uses them (and its use of globals() is
restricted to specific values so it really doesn't need to be quite so
broad). Basically I wished __import__() looked like
> Maybe two opcodes is not even enough. Maybe we should have one to
> resolve relative imports (i.e. import.c:resolve_name), one to load but
> not exec a module given its absolute name (i.e. _find_and_load()
> without the exec), one to exec a loaded module, one or more to handle
> the horror of "fromlist" (i.e. _handle_fromlist()).
I have always wanted to at least break up getting the module and fromlist
as separate opcodes, so +1 for that. Name resolution could potentially be
done as an opcode as it relies on execution state pulled from the globals
of the module, but the logic also isn't difficult so +0 for that (i.e.
making an opcode that calls something more like importlib.import_module()
is more critical to me than eliminating the 'package' argument to that
call, but I don't view it as a bad thing to have another opcode for that
As for the completely separating the loading and execution, I don't have a
need for what's being proposed so I don't have an opinion. I basically made
sure Eric Snow structured specs so that lazy loading as currently supported
works so I got what I wanted for basic lazy importing (short of the PyPI
package I keep talking about writing to add a nicer API around lazy
importing :) .
On Tue, Nov 28, 2017 at 12:31:06PM -0800, Raymond Hettinger wrote:
> > I also cc python-dev to see if anybody here is strongly in favor or against this inclusion.
> Put me down for a strong -1. The proposal would occasionally save a
> few keystokes but comes at the expense of giving Python a more Perlish
> look and a more arcane feel.
I think that's an unfair characterisation of the benefits of the PEP.
It's not just "a few keystrokes".
Ironically, the equivalent in Perl is // which Python has used for
truncating division since version 2.4 or so. So if we're in danger of
looking "Perlish", that ship has sailed a long time ago.
Perl is hardly the only language with null-coalescing operators -- we
might better describe ?? as being familiar to C#, PHP, Swift and Dart.
That's two mature, well-known languages and two up-and-coming languages.
> timeout ?? local_timeout ?? global_timeout
As opposed to the status quo:
timeout if timeout is not None else (local_timeout if local_timeout is not None else global_timeout)
Or shorter, but even harder to understand:
(global_timeout if local_timeout is None else local_timeout) if timeout is None else timeout
I'd much prefer to teach the version with ?? -- it has a simple
explanation: "the first of the three given values which isn't None". The
?? itself needs to be memorized, but that's no different from any other
operator. The first time I saw ** I was perplexed and couldn't imagine
what it meaned.
Here ?? doesn't merely save a few keystrokes, it significantly reduces
the length and complexity of the expression and entirely cuts out the
duplication of names.
If you can teach
timeout or local_timeout or global_timeout
then you ought to be able to teach ??, as it is simpler: it only
compares to None, and avoids needing to explain or justify Python's
> 'foo' in (None ?? ['foo', 'bar'])
If you can understand
'foo' in (False or ['foo', 'bar'])
then surely you can understand the version with ??.
> requested_quantity ?? default_quantity * price
(default_quantity if requested_quantity is None else requested_quantity) * price
I'd much prefer to read, write and teach the version with ?? over the