Mailman 3 May 2017 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

Specify number of items to allocate for array.array() constructor
by Sven Rahmann Feb. 22, 2020

Feb. 22, 2020

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int … [View More]

14 20

Positional-only parameters
by Victor Stinner Sept. 11, 2018

Sept. 11, 2018

Hi, For technical reasons, many functions of the Python standard libraries implemented in C have positional-only parameters. Example: ------- $ ./python Python 3.7.0a0 (default, Feb 25 2017, 04:30:32) >>> help(str.replace) replace(self, old, new, count=-1, /) # <== notice "/" at the end ... >>> "a".replace("x", "y") # ok 'a' >>> "a".replace(old="x", new="y") # ERR! TypeError: replace() takes at least 2 arguments (0 given) ------- When converting the … [View More]

19 27

Implicit string literal concatenation considered harmful?
by Guido van Rossum March 14, 2018

March 14, 2018

I just spent a few minutes staring at a bug caused by a missing comma -- I got a mysterious argument count error because instead of foo('a', 'b') I had written foo('a' 'b'). This is a fairly common mistake, and IIRC at Google we even had a lint rule against this (there was also a Python dialect used for some specific purpose where this was explicitly forbidden). Now, with modern compiler technology, we can (and in fact do) evaluate compile-time string literal concatenation with the '+' … [View More]

51 165

JavaScript-Style Object Creation in Python (using a constructor function instead of a class to create objects)
by Simon Ramstedt July 13, 2017

July 13, 2017

Hi, do you have an opinion on the following? Wouldn't it be nice to define classes via a simple constructor function (as below) instead of a conventional class definition? *conventional*: class MyClass(ParentClass): def __init__(x): self._x = x def my_method(y): z = self._x + y return z *proposed*: def MyClass(x): self = ParentClass() def my_method(y): z = x + y return z self.my_method = my_method # that's … [View More]cumbersome (see comments below) return self Here are the pros and cons I could come up with for the proposed method: (+) Simpler and more explicit. (+) No need to create attributes (like `self._x`) just to pass something from `__init__` to another method. (+) Default arguments / annotations for methods could be different for each class instance. Adaptive defaults wouldn't have to simulated with a None. (+) Class/instance level imports would work. (-/+) Speed: The `def`-based objects take 0.6 μs to create while the `class`-based objects take only 0.4 μs. For method execution however the closure takes only 0.15 μs while the proper method takes 0.22 μs (script <https://gist.github.com/rmst/78b2b0f56a3d9ec13b1ec6f3bd50aa9c>). (-/+) Checking types: In the proposed example above the returned object wouldn't know that it has been created by `MyClass`. There are a couple of solutions to that, though. The easiest to implement would be to change the first line to `self = subclass(ParentClass())` where the subclass function looks at the next item in the call stack (i.e. `MyClass`) and makes it the type of the object. Another solution would be to have a special rule for functions with capital first letter returning a single object to append itself to the list of types of the returned object. Alternatively there could be a special keyword e.g. `classdef` that would be used instead of `def` if we wouldn't want to rely on the name. (-) The current syntax for adding a function to an object is cumbersome. That's what is preventing me from actually using the proposed pattern. But is this really the only reason for not using it? And if so, wouldn't that be a good argument for enabling something like below? *attribute function definitions*: def MyClass(x): self = ParentClass() def self.my_method(y): z = x + y return z return self or alternatively *multiline lambdas*: def MyClass(x): self = ParentClass() self.my_method = (y): z = x + y return z return self Cheers, Simon [View Less]

25 80

SealedMock proposal for unittest.mock
by Mario Corchero June 1, 2017

June 1, 2017

Hello Everyone! First time writing to python-ideas. *Overview* Add a new mock class within the mock module <https://github.com/python/cpython/blob/master/Lib/unittest/mock.py>, SealedMock (or RestrictedMock) that allows to restrict in a dynamic and recursive way the addition of attributes to it. The new class just defines a special attribute "sealed" which once set to True the behaviour of automatically creating mocks is blocked, as well as for all its "submocks". See sealedmock <… [View More]

3 3

dict(default=int)
by Steven Piantadosi May 30, 2017

May 30, 2017

Hi All, I find importing defaultdict from collections to be clunky and it seems like having a default should just be an optional keyword to dict. Thus, something like, d = dict(default=int) would be the same as from collections import defaultdict d = defaultdict(int) Any thoughts? Thanks, ++Steve

15 19

a memory-efficient variant of itertools.tee
by Stephan Houben May 29, 2017

May 29, 2017

Hi all, The itertools.tee function can hold on to objects "unnecessarily". In particular, if you do iter2 = itertools.tee(iter1, 2)[0] i.e. you "leak" one of the returned iterators, then all returned objects are not collected until also iter2 is collected. I propose a different implementation, namely the one in: https://github.com/stephanh42/streamtee streamtee.tee is a drop-in alternative for itertools.tee but as you can see from the test in the repo, it will not hold on to the generated … [View More]

2 2

tweaking the file system path protocol
by Wolfgang Maier May 29, 2017

May 29, 2017

What do you think of this idea for a slight modification to os.fspath: the current version checks whether its arg is an instance of str, bytes or any subclass and, if so, returns the arg unchanged. In all other cases it tries to call the type's __fspath__ method to see if it can get str, bytes, or a subclass thereof this way. My proposal is to change this to: 1) check whether the type of the argument is str or bytes *exactly*; if so, return the argument unchanged 2) check wether __fspath__ … [View More]

9 24

Exposing CPython's subinterpreter C-API in the stdlib.
by Eric Snow May 27, 2017

May 27, 2017

Although I haven't been able to achieve the pace that I originally wanted, I have been able to work on my multi-core Python idea little-by-little. Most notably, some of the blockers have been resolved at the recent PyCon sprints and I'm ready to move onto the next step: exposing multiple interpreters via a stdlib module. Initially I just want to expose basic support via 3 successive changes. Below I've listed the corresponding (chained) PRs, along with what they add. Note that the 2 … [View More]

10 20

pdb / bdb adding watchpoint ability
by Louie Lu May 25, 2017

May 25, 2017

Hi all, Current pdb and bdb didn't have the ability to create a watchpoint for variable. Such in gdb, it have a series of watchpoint function*: * watch : break when expression value changed * rwatch : break when read value from expression * awatch: breka when read or write value to expression In Python, there have some problem in rwatch implement, that we can't directly know what variable is been read or not. This will need to used the dis module to get the bytecode and to … [View More]

1 0

Suggestion: push() method for lists
by Paul Laos May 22, 2017

May 22, 2017

It would be nice to have the opposite of the pop() method: a push() method. While insert() and append() already exist, neither is the opposite of pop(). pop() has a default index parameter -1, but neither insert() nor append() has a default index parameter. push(obj) would be equivalent to insert(index = -1, object), having -1 as the default index parameter. In fact, push() could replace both append() and insert() by unifying them. By default push(obj) would insert an object to the end of the … [View More]list, while push(obj, index) would insert the object at index, just like pop() removes (and returns) the object at the end of the list, and pop(index) removes (and returns) the object at the index. I found little discussion on this, just this SO thread http://stackoverflow.com/questions/1566266/why-is-pythons-append-not-push which lead to a discussion from 20 years ago (1997): <https://groups.google.com/forum/#!topic/comp.lang.python/SKJq3S2ZYm> https://groups.google.com/forum/#!topic/comp.lang.python/SKJq3S2ZYmg <https://groups.google.com/forum/#!topic/comp.lang.python/SKJq3S2ZYmg> Some key arguments from the thread: >it would be an >easy and obvious improvement to make popend an explicit builtin, and >that this would make Python even more attractive to newcomers who want >to use what they already know and be immediately productive. - Terry Reedy >append() is a special case of insert(). The inverse of insert() is >the del function. The specific inverse of append() is del list[-1]. - Michael W. Ryan >but I'm not a big fan of multiple names for the same operation -- >sooner or later you're going to read code that uses the other one, so >you need to learn both, which is more cognitive load. - Guido van Rossum So while it has been discussed before, it's worth bringing up again, since this was before the release of Python 2.0. Pros: - Would simplify the language by having a symmetric relation to pop(). - Would make it easy to use lists as stacks. - Saves at least two characters - If append()/insert() are being removed and replaced, the complexity of lists is slightly reduced. Cons: - Would blur the line between lists and stacks. - The order of the parameters in push(obj, index = -1) would be the opposite of the parameters in insert(index, obj), because defaulted parameters come last. - If append()/insert() are being removed and replaced, backwards compatability breaks. - If append()/insert() are kept instead of being replaced, the complexity of lists is slightly increased. While it isn't a necessity, I believe the benefit of push() method outweighs its cons. ~Paul [View Less]

5 5

fnmatch.filter_false
by Alex Walters May 21, 2017

May 21, 2017

Fnmath.filter works great. To remind people what it does, it takes an iterable of strings and a pattern and returns a list of the strings that match the pattern. And that is wonderful However, I often need to filter *out* the items that match the pattern (to ignore them). In every project that I need this I end up copying the function out of the fnmatch library and adding 'not' to the test clause. It would be wonderful if there was a filter_false version in the standard library. Or in … [View More]

7 15

Re: [Python-ideas] JavaScript-Style Object Creation in Python (using a constructor function instead of a class to create objects)
by Tin Tvrtković May 19, 2017

May 19, 2017

Hello, I'm an attrs contributor so maybe I can clear up any questions. Convert callables are only called in __init__, not in the setters. We've had this requested a number of times and we will almost certainly support it in the future, probably on an opt-in basis. The reason we don't currently support it is mostly technical. We try really hard for our features to not add significant overhead to the generated classes, and doing this with minimal overhead for slot classes basically requires C/… [View More]Cython or attribute *access* becomes significantly slower. But this is implementation stuff and not pertinent here. Date: Fri, 19 May 2017 20:49:08 +0200 > From: Stephan Houben <stephanh42(a)gmail.com> > To: guido(a)python.org > Cc: "Eric V. Smith" <eric(a)trueblade.com>, Python-Ideas > <python-ideas(a)python.org> > Subject: Re: [Python-ideas] JavaScript-Style Object Creation in Python > (using a constructor function instead of a class to create objects) > Message-ID: > <CAOOa=pOBk8i1+sy+sZ4C+oLwhZzhLAjNRNiaEh8N9Srv= > pxDNw(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Guido, > > Yes indeed, *only* invoked by __init__ . > > See my test below. > ===== > import attr > > @attr.s > class Foo: > x = attr.ib(convert=str) > > foo = Foo(42) > print(repr(foo.x)) > # prints '42' > foo.x = 42 > print(repr(foo.x)) > # prints 42 > ====== > > Not sure if this is a good design but it matches the docs. > > Stephan > > Op 19 mei 2017 20:36 schreef "Guido van Rossum" <guido(a)python.org>: > > So it is only called by __init__ and not by __setattr__? > > On Fri, May 19, 2017 at 11:32 AM, Stephan Houben <stephanh42(a)gmail.com> > wrote: > > > Let me quote the attrs docs: > > > > "" > > convert (callable) ? callable() that is called by attrs-generated > __init__ > > methods to convert attribute?s value to the desired format. It is given > the > > passed-in value, and the returned value will be used as the new value of > > the attribute. The value is converted before being passed to the > validator, > > if any. > > """ > > > > So the signature is essentially: > > > > self.myattrib = callable (myattrib) > > > > Stephan > > > > Op 19 mei 2017 20:25 schreef "Guido van Rossum" <guido(a)python.org>: > > > > For people who don't want to click on links: > >> > >> 1. Allow hash and equality to be based on object identity, rather than > >> structural identity, > >> this is very important if one wants to store un-hashable objects in > >> the instance. > >> (In my case: mostly dict's and numpy arrays). > >> > >> 2. Not subclassed from tuple. I have been bitten by this subclassing > >> when trying to set up > >> singledispatch on sequences and also on my classes. > >> > >> 3. Easily allow to specify default values. With namedtuple this > >> requires overriding __new__. > >> > >> 4. Easily allow to specify a conversion function. For example I have > >> some code like below: > >> note that I can store a numpy array while keeping hashability and > >> I can make it convert > >> to a numpy array in the constructor. > >> > >> @attr.s(cmp=False, hash=False) > >> class SvgTransform(SvgPicture): > >> child = attr.ib() > >> matrix = attr.ib(convert=numpy.asarray) > >> > >> > >> I have one question about (4) -- how and when is the conversion > function used, and what is its signature? > >> > >> > >> On Fri, May 19, 2017 at 5:42 AM, Eric V. Smith <eric(a)trueblade.com> > >> wrote: > >> > >>> Could you point me to this 4-point list of Stephan's? I couldn't find > >>>> anything in the archive that you might be referring to. > >>>> > >>> > >>> Never mind, I found them here: > >>> https://mail.python.org/pipermail/python-ideas/2017-May/045679.html > >>> > >>> Eric. > >>> > >>> > >>> _______________________________________________ > >>> Python-ideas mailing list > >>> Python-ideas(a)python.org > >>> https://mail.python.org/mailman/listinfo/python-ideas > >>> Code of Conduct: http://python.org/psf/codeofconduct/ > >>> > >> > >> > >> > >> -- > >> --Guido van Rossum (python.org/~guido) > >> > >> _______________________________________________ > >> Python-ideas mailing list > >> Python-ideas(a)python.org > >> https://mail.python.org/mailman/listinfo/python-ideas > >> Code of Conduct: http://python.org/psf/codeofconduct/ > >> > >> > > [View Less]

1 0

PEP 526: why ClassVar instead of ClassAttr?
by Steven D'Aprano May 18, 2017

May 18, 2017

Since PEP 526 is already provisionally accepted, it may be too late to bring this up, but I have a question and suggestion about the name ClassVar. I've read the PEP but didn't see an answer or rejection to this. https://www.python.org/dev/peps/pep-0526/ Why choose ClassVar over ClassAttr when the usual terminology used in the Python community is class and instance *attributes* rather than "variables"? I understand that, in a sense, attributes are variables (unless they're constants *… [View More]

6 8

Tighten up the formal grammar and parsing a bit?
by Hugh Fisher May 18, 2017

May 18, 2017

I wrote this little Python program using CPython 3.5.2. It's ... interesting ... that we apparently don't need comments or pass statements any more. Anyone else think it might be worth tightening up the grammar definition and parser a bit? def empty(): """Don't do anything""" def helloWorld(): """Docstring""" x = 0 if x > 0: """Pass""" else: x += 1 print(x) """Comment that is a string or vice versa""" x = 2 print(x) if x == 2: … [View More]

6 9

Running C extension modules using -m switch
by gmarcel.plch＠gmail.com May 17, 2017

May 17, 2017

Greetings, I'm a student that has been working lately on feature of the runpy module that I have been quite interested in: execution of extension modules using the -m switch. Currently this requires access to the module's code, so it only works for modules written in Python. I have a proof-of-concept implementation that adds a new ExtensionFileLoader method called "exec_as_main". The runpy module then checks if the loader has this method, and if so, calls it instead of getting the the code and … [View More]

1 0

Re: [Python-ideas] Python-ideas Digest, Vol 126, Issue 35
by Tin Tvrtković May 16, 2017

May 16, 2017

I have it on good authority both Hynek (the author of attrs) and Glyph will be attending PyCon. I think it'd be a shame if they weren't involved with this effort somehow. > Message: 2 > Date: Tue, 16 May 2017 07:53:50 -0700 > From: Guido van Rossum <guido(a)python.org> > To: Ivan Levkivskyi <levkivskyi(a)gmail.com> > Cc: ????? <elazarg(a)gmail.com>, "python-ideas(a)python.org" > <python-ideas(a)python.org> > Subject: Re: [Python-ideas] … [View More]

2 1

NamedTuple Interface
by Fabrizio Messina May 15, 2017

May 15, 2017

I think it would be nice to have a generic NamedTuple interface in python: from typing import NamedTupleType def test( arguments: NamedTupleType ) -> NamedTupleType: return SomeType(**NamedTupleType._asdict) The rationale is that named tuple exposes a common API, and it would be nice to have it readily available.

1 0

Re: [Python-ideas] singledispatch for instance methods
by Bar Harel May 14, 2017

May 14, 2017

Bump On Wed, Jan 4, 2017, 8:01 PM Lisa Roach <lisaroach14(a)gmail.com> wrote: > +1 to this as well, I think this would be really useful in the stdlib. > > On Mon, Dec 26, 2016 at 5:40 AM, Bar Harel <bzvi7919(a)gmail.com> wrote: > >> Any updates with a singledispatch for methods? >> >> On Tue, Sep 20, 2016, 5:49 PM Bar Harel <bzvi7919(a)gmail.com> wrote: >> >>> At last! Haven't used single dispatch exactly because of that. Thank … [View More]

4 9

add a LogReader to the logging module
by terji78＠gmail.com May 11, 2017

May 11, 2017

I get a message back that I'm not subscribed to the mailing list, but see my message in google groups. My sincerest apologies in advance, if this appears several times for you. Anyways: The logging module has an easy-to-setup logger: import logging logger = logging.getLogger(__name__) and you just log away. Easy. However, it's quite a bit more difficult to set up log readers, requiring IMO an unreasonably number of lines of code: import logging logger = logging.… [View More]

3 3

Add shutil.ignore_patterns() to shutil.rmtree()
by George Fischhof May 10, 2017

May 10, 2017

Hi Folks, I have a task to synchronize folders but some files should be remained untouched. I think this is a very common task. I found that shutil.copytree() has ignore_patterns() but rmtree() has not. So here comes my idea: add ignore_patterns() to rmtree() it is a good feature and makes the functions symmetric. BR, George

3 8

Add a .chunks() method to sequences
by robert.hoelzl＠posteo.de May 5, 2017

May 5, 2017

Steven D’Aprano was giving me an idea (in the bytes.hex delimiter discussion): I had very often the use case that I want to split sequences into subsequences of same size. How about adding a chunks() and rchunks() function to sequences: [1,2,3,4,5,6,7].chunks(3) => [[1,2,3], [4,5,6], [7]] "1234“.chunks(2) => [“12“, “34“] (this could then be used to emulate stevens proposal: “ “.join(“1234567“.chunks(2)) => “12 34 56 7“) robert

6 7

Storing a reference to the function object (if any) inside frame objects
by Nathaniel Smith May 4, 2017

May 4, 2017

Hi all, Currently, given a frame object (e.g. from sys._getframe or inspect.getouterframes), there's no way to get back to the function object that created it. This creates an obstacle for various sorts of introspection. In particular, in the unusual but real situation where you need to "mark" a function in a way that can be detected via stack introspection, then the natural way to do that – attaching an attribute to the function object – doesn't work. Instead, you have to mangle the frame's … [View More]locals. Example: In pytest, if you want to mark a function as uninteresting and not worth showing in tracebacks (a common situation for assertion helpers), then you can't do this with a decorator like @pytest.tracebackhide; instead, you have to set a __magic__ local variable: https://docs.pytest.org/en/latest/example/simple.html#writing-well-integrat… Example: In trio, I want to mark certain Python functions as "protected" from receiving KeyboardInterrupt exceptions – kind of like how CPython doesn't allow KeyboardInterrupt in internal functions [1]. If it were possible for my signal handler to get at function objects when it walks the stack, then the necessary decorator would be something like: def keyboard_interrupt_protected(fn): fn._trio_keyboard_interrupt_protected = True return fn Instead right now it's this mess: https://github.com/python-trio/trio/blob/64119b12309ffeaf3a35622ef08d3b03e4… And worse, the current mess has a race condition, because a KeyboardInterrupt could arrive while we're in the middle of executing the code to enable the KeyboardInterrupt protection. OTOH if this information was carried on the function object, the protection would be be automatically set up when the frame was created. Another benefit is that it would make it easy to include __qualname__'s in exception tracebacks, which is often nice. For example, here's a current traceback: File "/home/njs/trio/trio/_sync.py", line 374, in acquire_nowait return self._lock.acquire_nowait() File "/home/njs/trio/trio/_sync.py", line 277, in acquire_nowait raise WouldBlock And here's what it could look like if the traceback machinery could access frame.f_func.__qualname__: File "/home/njs/trio/trio/_sync.py", line 374, in Condition.acquire_nowait return self._lock.acquire_nowait() File "/home/njs/trio/trio/_sync.py", line 277, in Lock.acquire_nowait raise WouldBlock Here's some semi-terrifying code attempting to work around this issue: https://stackoverflow.com/questions/14817788/python-traceback-with-module-n… To be fair, this last issue could also be solved by adding a f_qualname field to frame objects, like generators and coroutines already have. But another way to think about it is that if we had the whole function object then generators and coroutines wouldn't have to track the qualname separately anymore :-). Q: But what about frames that don't have any associated function object, like those created by exec()? A: Then frame.f_func would be NULL/None, no biggie. Q: Won't this increase memory usage? A: In principle this could increase memory usage by keeping function objects alive longer than otherwise, but I think the effect should be pretty minimal. It doesn't introduce any reference loops, and the vast majority of function objects outlive their frames in any case. Specifically I think the only (?) case where it would make a difference is if you create a new function and then call it immediately without storing it in a temporary, like: (lambda: 1)(). So basically: it's a bit obscure, but I think would be a nice little addition and I don't see any major downsides. Anything I'm missing? -n [1] There's a lot more detail on this here: https://vorpus.org/blog/control-c-handling-in-python-and-trio/ but it's probably unnecessary for the current discussion. -- Nathaniel J. Smith -- https://vorpus.org [View Less]

2 1

Add an option for delimiters in bytes.hex()
by robert.hoelzl＠posteo.de May 4, 2017

May 4, 2017

The bytes.hex() function is the inverse function of Bytes.fromhex(). But fromhex can process spaces (which is much more readable), while hex() provides no way to include spaces. My proposal would be to add an optional delimiter, that allows to specify a string that will be inserted between the digit pairs of a byte: def hex(self, delimiter=‘‘): … This would allow to write: assert b’abc‘.hex(‘ ‘) == ’61 62 63‘ Gesendet von Mail für Windows 10

13 30