Mailman 3 December 2014 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

Specify number of items to allocate for array.array() constructor
by Sven Rahmann Feb. 22, 2020

Feb. 22, 2020

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int … [View More]

14 20

Implicit string literal concatenation considered harmful?
by Guido van Rossum March 14, 2018

March 14, 2018

I just spent a few minutes staring at a bug caused by a missing comma -- I got a mysterious argument count error because instead of foo('a', 'b') I had written foo('a' 'b'). This is a fairly common mistake, and IIRC at Google we even had a lint rule against this (there was also a Python dialect used for some specific purpose where this was explicitly forbidden). Now, with modern compiler technology, we can (and in fact do) evaluate compile-time string literal concatenation with the '+' … [View More]

51 165

pep 397
by Liam Marsh Feb. 23, 2015

Feb. 23, 2015

hello, *the pep 397 says that any python script is able to choose the language version which will run it, between all the versions installed on the computer, using on windows a launcher in the "C:\windows" folder.* can the idle version be chosen like this too, or can the idle "run" command do it? (except it is already like this and I have a problem) thank you, and have a nice day/evening! (and sorry if this sounds irritating)

6 14

python on mobile
by Fetchinson . Jan. 28, 2015

Jan. 28, 2015

Hi all, maybe the best list would be python-dev but I don't dare waking the sleeping lions there :) So my question is this: is there a coherent mobile strategy among core dev people? I mean you guys release python for linux/macos/windows and the question is if there are any plans to do the same for a mobile platform. It doesn't have to be android or ios just anything that the core dev team chooses and sticks with. I've been developing python apps for smartphones (mostly hobby projects though) … [View More]

14 33

Extending traceback to (optionally) format and show locals.
by Robert Collins Jan. 18, 2015

Jan. 18, 2015

For context please see http://bugs.python.org/issue22937 and http://bugs.python.org/issue22936. I have two questions I'm hoping to get answered through this thread: - does the change in question need a PEP? Antoine seemed to think it didn't, as long as it was opt-in (via the unittest CLI etc) - Is my implementation approach sound (for traceback, unittest I think I have covered :))? Implementation wise, I think its useful to work in the current traceback module layout - that is to alter … [View More]

4 7

Adding `pathlib.Path` method that would send file to recycle bin
by Ram Rachum Jan. 7, 2015

Jan. 7, 2015

Hi guys, I want to have a method of `pathlib.Path` that would send a file to the recycle bin. (i.e. soft delete.) What do you think about adding this? I see there's a PyPI package `Send2Trash` that does this, but it would be nice if this was in the standard library. Thanks, Ram.

24 120

Re: [Python-ideas] weakref.WeakKeyDictionary is (basically) useless
by Kevin Norris Dec. 31, 2014

Dec. 31, 2014

On Wed, Dec 31, 2014 at 6:00 AM, Antoine Pitrou <solipsis(a)pitrou.net> wrote: >> Why would you use it? The official documentation mentions >> implementing a [descriptor][1]. See [this paste][2] for a simple >> example of this sort of thing. > > How about something like this (untested): > A cursory glance suggests that might work for my use-case, but I would want to examine it more thoroughly for race conditions and other hazards. In particular, I'd be … [View More]

1 0

Python hook just before NameError
by Chris Angelico Dec. 31, 2014

Dec. 31, 2014

I mainly did up this patch to see how hard it would be, and now it's turned out to be fairly simple, I'm curious as to whether it would actually be useful to people. At the point where a global/builtin name lookup is about to raise NameError, first try calling a Python function, along the same lines as __getattr__. If that function hasn't been defined, raise NameError as normal; but if it has, let it either raise NameError or return some object, which is then used as if the name had been bound … [View More]

8 15

weakref.WeakKeyDictionary is (basically) useless
by Kevin Norris Dec. 31, 2014

Dec. 31, 2014

Let's talk about weakref.WeakKeyDictionary. First, what is it? It's a dictionary whose keys are referenced weakly. That is, the dictionary takes weak references to its keys. If a key is garbage collected, it magically vanishes from the dictionary. This saves programmers much of the trouble of manually culling dead weak references from data structures. Why would you use it? The official documentation mentions implementing a [descriptor][1]. See [this paste][2] for a simple example of this … [View More]sort of thing. Turning the WeakKeyDictionary into a regular dictionary would be troublesome. Since the descriptor object will (probably) never die, if it used a normal dictionary, it would keep all/most/some/more-than-one instances of Bar() alive forever. While this simple example would be better implemented as a property, or even a bare attribute, it is nice if you have many separate Bar classes and complicated per-property processing. Why is it broken? To go back to our simple example, imagine someone subclasses Bar (let's call the subclass Baz) and implements __eq__() and/or __hash__(). Instances of Baz may not work at all (if __hash__ is None) or they may "work" but behave nonsensically (all values-equal instances of Baz will have the same .foo attribute - imagine trying to track that bug down!). While we could forbid such a subclass in our API docs, this runs afoul of the open/closed principle. What can we do about it? It's possible someone is relying on this behavior, so we probably should not change the existing class. Instead, we can offer a new class called, say, WeakKeyIDDictionary. Said class would use object identity instead of object equality/hashing to uniquify keys. It would also reference keys weakly, just like WeakKeyDictionary does now. Implementing such a class is somewhat tricky, since weak references are difficult to reason about correctly. I've written up a [simplistic prototype][3], but I'm concerned about both performance and correctness. Specifically, I know that I need to call _remove_dead_refs() periodically, but I don't see where I can do that without compromising performance (e.g. making key lookup O(n) is a Bad Idea). I also would note it looks nothing like the stdlib implementations of WeakKey/ValueDictionary, which finagle raw weakrefs directly. These problems are related, of course, since the stdlib uses finalizer callbacks to cull dead references as they die. Frankly, the stdlib's code looks incredibly hairy to me and I doubt I could correctly re-implement it. Finally, I've no idea if it's subject to race conditions or other issues (it is a completely untested example I threw together in 5 minutes). Because this class is simultaneously useful and hard to write correctly, it would be a nice addition to the standard library (probably not in the form I've written it!), or failing that, some other library. I've Googled for such a library and come up empty. Should I submit a bug, a PEP, or is this whole idea just stupid? [1]: https://docs.python.org/3/reference/datamodel.html#implementing-descriptors [2]: http://pastebin.com/90j2ZAqC [3]: http://pastebin.com/cpZh2bTa -- Kevin Norris [View Less]

3 2

Re: [Python-ideas] weakref.WeakKeyDictionary is (basically) useless
by Kevin Norris Dec. 31, 2014

Dec. 31, 2014

On Tue, Dec 30, 2014 at 9:33 PM, Nils Bruin <bruin.nils(a)gmail.com> wrote: > We ran into this for sage (www.sagemath.org) and we implemented > "MonoDict" with basically the semantics you describe. It is > implemented as a cython extension class. See: > > http://git.sagemath.org/sage.git/tree/src/sage/structure/coerce_dict.pyx#n2… > > I can confirm your observation that it's hard to get right, but I think our > implementation by now is both correct and fast. The … [View More]

1 0

Re: [Python-ideas] weakref.WeakKeyDictionary is (basically) useless
by Kevin Norris Dec. 31, 2014

Dec. 31, 2014

On Tue, Dec 30, 2014 at 8:34 PM, Benjamin Peterson <benjamin(a)python.org> wrote: > Why not just use a wrapper like this for keys? > > class IdentityWrapper(object): > def __init__(self, obj): > self.obj = obj > def __hash__(self): > return id(self.obj) > def __eq__(self, other): > return self.obj is other I considered an approach like this one, but couldn't quite make it work. How do we make the … [View More]

1 0

Re: [Python-ideas] Python hook just before NameError
by Steven D'Aprano Dec. 30, 2014

Dec. 30, 2014

On Mon, Dec 29, 2014 at 08:39:41AM -0800, Rick Johnson wrote: > > > On Monday, December 29, 2014 8:12:49 AM UTC-6, Steven D'Aprano wrote: > > I just threw this lazy import proxy object together. I haven't tested it > > extensively, but it seems to work: [...] > All you've done is to replace "import decimal" with "decimal = > lazy_import('decimal')" -- what's the advantage? It delays the actual import of the module until you try to use it. For decimal, that's … [View More]

3 2

Change Behind The Scenes Attribute Access from Class to Object
by Adam Carruthers Dec. 29, 2014

Dec. 29, 2014

At the moment, when a python function, like print, calls an object attribute it does this (if it were written in python): type(obj).__str__(obj)This can be restrictive in lots of situations and I think it would be better to just get the attribute from the object itself. obj.__str__()It would make you free to override it on a per object basis. In some cases, this could lead to huge optimization.Obviously, it is easy to fix yourself, but it is a layer of uncharacteristic unintuitiveness: def __str__(self): self._str()

2 1

Fwd: python on mobile
by Russell Keith-Magee Dec. 29, 2014

Dec. 29, 2014

On Mon, Dec 29, 2014 at 8:09 AM, Steven D'Aprano <steve(a)pearwood.info> wrote: > > On Sun, Dec 28, 2014 at 10:30:41PM +0100, Fetchinson . wrote: > > On 12/27/14, Russell Keith-Magee <russell(a)keith-magee.com> wrote: > > > > This is an entirely > > > reasonable approach if Kivy is what you want at the end of the day - > but > > > it's not what *I* want. I've been taking a much more atomic approach. > The > > > Kivy toolchain … [View More]

1 0

Type Hinting - Performance booster ?
by Ludovic Gasc Dec. 27, 2014

Dec. 27, 2014

Hi, I'm only a simple Python developer, not a Type Hinting expert and I don't know if you already discuss about that somewhere, but: With the future official support of Type Hinting in Python, is it means that CPython could use this pieces of information to store variables with more efficient data structures, not only check types ? It could possible to have better performance one day with Type Hinting like you have with Cython (explicit types declaration) or PyPy (guessing types) ? Is it … [View More]

14 34

Type Hinting Kick-off
by Guido van Rossum Dec. 27, 2014

Dec. 27, 2014

A few months ago we had a long discussion about type hinting. I've thought a lot more about this. I've written up what I think is a decent "theory" document -- writing it down like this certainly helped *me* get a lot of clarity about some of the important issues. https://quip.com/r69HA9GhGa7J I should thank Jeremy Siek for his blog post about Gradual Typing, Jukka Lehtosalo for mypy (whose notation I am mostly borrowing), and Jim Baker for pushing for an in-person meeting where we all got a … [View More]

11 35

Fwd: improve compatibility
by Liam Marsh Dec. 23, 2014

Dec. 23, 2014

Hello again a few days later, i wad a new idea: the user who wants the 2.X modules, the 3.2 and 3.4 modules to work will probably install the three versions. so, is it possible to have the <somepath>\Python\python34; <somepath>\Python\python32; and <somepath>\Python\python27 dirs, and the <somepath>\Python\PythonAutoVersion program, useing the comment "#py ver XX" to chose which version will take care of it? (same thing with the IDLE) how about that? have a nice day/evening!

4 6

Bug? Feature? setattr(foo, '3', 4) works!
by Cem Karan Dec. 22, 2014

Dec. 22, 2014

I accidentally discovered that the following works, at least in Python 3.4.2: >>> class foo(object): ... pass ... >>> setattr(foo, '3', 4) >>> dir(foo) ['3', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '… [View More]

4 14

Line-by-line profiling
by Todd Dec. 20, 2014

Dec. 20, 2014

Currently, the python profiler works at the function or method level. This is good for identifying slow functions, but not so good for identifying things like slow for loops inside larger functions. You can use timers, but putting a lot of times in its cumbersome and makes direct comparison between potential hotspots difficult. And although we should, in practice it isn't airways feasible to refactor or code so there is only one major hotspot per function or method. A solution to this is to … [View More]

2 1

Re: [Python-ideas] Type Hinting Kick-off -- overloading
by Guido van Rossum Dec. 20, 2014

Dec. 20, 2014

On Fri, Dec 19, 2014 at 5:36 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > > > On 20 December 2014 at 10:55, Guido van Rossum <guido(a)python.org> wrote: > >> A few months ago we had a long discussion about type hinting. I've >> thought a lot more about this. I've written up what I think is a decent >> "theory" document -- writing it down like this certainly helped *me* get a >> lot of clarity about some of the important issues. >> >&… [View More]

2 1

A service to crawl +1s and URLs out of mailman archives
by Wes Turner Dec. 19, 2014

Dec. 19, 2014

In context to building a PEP or similar, I don't know how many times I've trawled looking for: * Docs links * Source links * Patch links * THREAD POST LINKS * Consensus A tool to crawl structued and natural language data from the forums could be very useful for preparing PEPs.

4 15

Why does BoundArguments use an OrderedDict?
by Antony Lee Dec. 19, 2014

Dec. 19, 2014

Currently, BoundArguments stores the results of signature.bind (and bind_partial) in an OrderedDict. Given that there is no way to retrieve the order in which keyword arguments were passed, it is not surprising that the dict ordering is based on the order of parameters in the signature, not the order in which arguments are passed (although this point could certainly be made clearer in the docs). But this also means that that ordering could easily be retrieved by iterating over the parameters (… [View More]

11 26

Generators are iterators
by Oscar Benjamin Dec. 16, 2014

Dec. 16, 2014

Somehow during the discussion of PEP 479 it was concluded that generators will become distinct from iterators. A few people have pointed out that this is a harmful distinction (complicates the language hard to explain etc). I think it is simply an invalid distinction and should be abolished. PEP 479 proposes to change the way that generators handle StopIteration fall-through by default. However the iterator protocol has never formally specified that an iterator should allow StopIteration to … [View More]fall through to a grandparent consumer. I considered the fact that StopIteration fall-through was possible to be a convenient design feature but it was never a required part of the definition of an iterator. Many iterators perform other kinds of exception handling without allowing that to fall through and many iterators catch StopIteration from child iterators without reraising it. This has never lead anyone to suggest that such iterators are not true iterators in any way. Under PEP 479 generators will handle a StopIteration arising from the executing frame differently. The generator will still be an object with __iter__ and __next__ that exposes StopIteration at the appropriate time to its parent iterator-consumer. In other words it will still satisfy the definition of an iterator. The PEP says: """ Under this proposal, generators and iterators would be distinct, but related, concepts. Like the mixing of text and bytes in Python 2, the mixing of generators and iterators has resulted in certain perceived conveniences, but proper separation will make bugs more visible. """" This is just plain false. Under the proposal generators will still be iterators. Mixing generators and iterators is nothing like mixing text and bytes and never has been. Mixing iterators and iterables is a bit like mixing text and the bytes in the sense that it can seem to work but then sometimes silently do the wrong thing. Mixing generators and iterators is like mixing sets and containers: there is no mixing since it is simply a false distinction. AFAICT from reading through the discussions this idea has (implicitly) followed from the following fallacious argument: 1) Bare next creates a problem for generators. 2) Therefore we fix it by changing generators. 3) Therefore generators are not iterators any more. Anyone with experience in propositional logic can see multiple fallacies in that argument but actually the biggest mistake is simply in the opening premise: bare next is a problem for all iterators. Generators are affected precisely because they are iterators. Generators were introduced as "a kind of Python iterator, but of an especially powerful kind" (PEP 255). The coroutine idea has since turned generators into something of a Frankenstein concept but the fundamental fact that they are iterators remains unchanged. And regardless of how much discussion coroutines generate the fact remains that 99% of generators are used purely for iteration. I propose to abolish this notion that generators are not iterators and to amend the text of the PEP to unambiguously state that generators are iterators regardless of any changes to the way they propagate StopIteration from the executing frame. Oscar [View Less]

16 60

Less tight validation in namedtuple input validation, and usefulness of issue 21832.
by Antony Lee Dec. 16, 2014

Dec. 16, 2014

While namedtuples have a good reason to not support field names starting with an underscore (namely, to prevent name collision), it would be nice to still be able to define some sort of "private" attribute using (manual) double-underscore name mangling, i.e. allow "namedtuple('Foo', 'bar baz _Foo__qux')". It used to be possible to bypass the validation by passing a carefully crafted subclass of str to the factory function, but this is no longer an option after the fix for issue 21832 (which, … [View More]

1 0