Mailman 3 July 2017 - Python-ideas

Re: [Python-ideas] namedtuple redesign goals
by Ethan Furman 25 Jul '17

25 Jul '17

[redirecting back to list] On 07/24/2017 04:19 AM, Michel Desmoulin wrote: > Le 24/07/2017 à 13:02, Ethan Furman a écrit : >> On 07/23/2017 10:47 AM, Michel Desmoulin wrote: >>> I'm not sure why everybody have such a grip on the type. >> >> If I understand the goal of "a new namedtuple" correctly, it is not to >> come up with yet another namedtuple type -- it is to make the existing >> collections.namedtuple a faster experience, and possibly add another way >> to create such a thing. >> >> This means that the "replacement" namedtuple MUST be backwards >> compatible with the existing collections.namedtuple, and keeping track >> of type is one of the things it does: > > Is it ? Maybe we should check that, cause we may be arguing around a "nice to have" for nothing. Um, yes, it is. Did you not read the section you snipped? [1] > How many people among those intereted by the proposal have a strong need for the type ? Whether there is a strong need for it is largely irrelevant; it's there now, it needs to stay. If we were to remove it there would need to be a strong need for what we gain and that it outweighs the broken backward compatibility commitment that we try very hard to maintain. -- ~Ethan~ [1] My apologies for the first paragraph if this is a language translation issue and you were talking about the backwards compatibility and not the type tracking.

3 3

namedtuple redesign goals
by Ethan Furman 24 Jul '17

24 Jul '17

On 07/23/2017 10:47 AM, Michel Desmoulin wrote: > I'm not sure why everybody have such a grip on the type. If I understand the goal of "a new namedtuple" correctly, it is not to come up with yet another namedtuple type -- it is to make the existing collections.namedtuple a faster experience, and possibly add another way to create such a thing. This means that the "replacement" namedtuple MUST be backwards compatible with the existing collections.namedtuple, and keeping track of type is one of the things it does: --> from collections import namedtuple --> Point = namedtuple('Point', 'x y') --> p1 = Point(3, 7) --> p1.x 3 --> p1.y 7 --> isinstance(p1, Point) True -- ~Ethan~

1 0

namedtuple from ordereddict
by Jim J. Jewett 21 Jul '17

21 Jul '17

Several of the replies seemed to suggest that re-using the current dict structure for a tuple wouldn't work. Since I'm not sure whether people are still thinking of the old structure, or I'm missing something obvious, I'll be more explicit. Per https://github.com/python/cpython/blob/master/Include/dictobject.h#L40 the last element of a dict object is now a separate pointer to ma_values, which is an array of objects. Per https://github.com/python/cpython/blob/master/Include/tupleobject.h#L27 the last element of a tuple object is also an array of objects. Is there some reason these arrays cannot be the same memory? e.g., does a tuple header *have* to be contiguous with its data, and if so, is there a reason that the dict's ma_array can't be allocated with an extra tuple-header prefix? -jJ

2 1

Fwd: Consider allowing the use of abstractmethod without metaclasses
by INADA Naoki 21 Jul '17

21 Jul '17

Hi, Neil. > I want to use abstractmethod, but I have my own metaclasses and I don't want > to build composite metaclasses using abc.ABCMeta. > > Thanks to PEP 487, one approach is to facator out the abstractmethod checks > from ABCMeta into a regular (non-meta) class. So, my first suggestion is to > split abc.ABC into two pieces, a parent regular class with metaclass "type": > I'm +1 with your idea in performance point of view. Some people having other language background (C# or Java) want to use ABC like Java's interface. But ABC is too heavy to use only for checking abstract methods. It uses three inefficient WeakSet [1] and it overrides isinstance and issubclass with slow Python implementation. [1] WeakSet is implemented in Python, having one __dict__, list and two sets. And C implementation of gathering abstract methods will reduce Python startup time too. Because `_collections_abc` module has many ABCs and `os` module import it. ## in abc.py # Helper function. ABCMeta use this too. # And Python 3.7 can have C implementation of this. def _init_abstractclass(cls, bases=None): # Compute set of abstract method names if bases is None: bases = cls.__bases__ abstracts = {name for name, value in vars(cls).items() if getattr(value, "__isabstractmethod__", False)} for base in bases: for name in getattr(base, "__abstractmethods__", set()): value = getattr(cls, name, None) if getattr(value, "__isabstractmethod__", False): abstracts.add(name) cls.__abstractmethods__ = frozenset(abstracts) class Abstract: __init_subclass__ = _init_abstractclass ## usage import abc class AbstractBar(abc.Abstract): @abc.abstractmethod def bar(self): ... Bests, INADA Naoki <songofacandy(a)gmail.com>

5 9

a new namedtuple
by Ethan Furman 20 Jul '17

20 Jul '17

Guido has decreed that namedtuple shall be reimplemented with speed in mind. I haven't timed it (I'm hoping somebody will volunteer to be the bench mark guru), I'll offer my NamedTuple implementation from my aenum [1] library. It uses the same metaclass techniques as Enum, and offers doc string and default value support in the class-based form. -- ~Ethan~ [1] https://pypi.python.org/pypi/aenum/1.4.5

10 17

Enabling Event.set to notify all waiters with an exception
by Pau Freixes 19 Jul '17

19 Jul '17

Morning guys, I came across with that idea trying to resolve a typical dogpile pattern [1], having many DNS calls to the same domain because of a miss in a DNS cache. The usage of the the set either to notify that the waiters could be awake and get the result from the cache or use it to notify that something was wrong helped me to reduce the complexity of the code. Just as an example: ``` if key in throttle_dns_events: yield from throttle_dns_events[key].wait() else: throttle_dns_events[key] = Event(loop=loop) try: addrs = yield from \ resolver.resolve(host, port, family=family) cached_hosts.add(key, addrs) throttle_dns_events[key].set() except Exception as e: # any DNS exception, independently of the implementation # is set for the waiters to raise the same exception. throttle_dns_events[key].set(exc=e) raise finally: throttle_dns_events.pop(key) ``` Any error caught by the locker will be broadcasted to the waiters. For example, a invalid hostname. I tried to open a PR to the CPython implementation, and they claim that the current interface of all of the locks objects behind the asyncio.locks [2] module try to keep the same interface as the threading one [3]. Therefore, to modify the asyncio implementation would need first a change in the threading interface. I was determined to justify that change, but after a bit research, I didn't find any example in other languages such as Java [4], C# [5] or C++ [6] allowing you to send an exception as a signal value to wake up the sleeping threads. is that enough to give up? I'm still reticent, I believe that this simple change in the interface can help reducing the complexity to handle errors in some scenarios. I would like to gather more ideas, thoughts, and comments from you about how can I still justify this change ... Thanks, [1] https://github.com/pfreixes/aiohttp/blob/throttle_dns_requests/aiohttp/conn… [2] https://docs.python.org/3/library/asyncio-sync.html [3] https://docs.python.org/3/library/threading.html [4] https://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#notifyAll() [5] https://msdn.microsoft.com/en-us/library/system.threading.monitor.pulseall.… [6] http://en.cppreference.com/w/cpp/thread/condition_variable/notify_all -- --pau

3 5

namedtuple with ordereddict
by Jim J. Jewett 19 Jul '17

19 Jul '17

Given that (1) dicts now always pay the price for ordering (2) namedtuple is being accelerated is there any reason not to simply define it as a view on a dict, or at least as a limited proxy to one? Then constructing a specific instance from the arguments used to create it could be as simple as keeping a reference to the temporary created to pass those arguments... -jJ

8 8

pathlib.Path should handle Pythonpath or package root
by George Fischhof 19 Jul '17

19 Jul '17

Hi there, I created a program which uses plugins (import them). I started to test it, and found that I need two types of paths: one for file system and another one which is package relative. So I thing this is a good idea, to enhance pathlib to handle package roots. (I know about sys.path, environment variables, importlib etc, but I think it should be in pathlib.) BR, George

2 5

Alternative Unicode implementations (NSString/NSMutableString)
by Jim J. Jewett 19 Jul '17

19 Jul '17

Ronald Oussoren came up with a concrete use case for wanting the interpreter to consider something a string, even if it isn't implemented with the default datastructure. In https://mail.python.org/pipermail/python-ideas/2017-July/046407.html he writes: The reason I need to subclass str: in PyObjC I use a subclass of str to represent Objective-C strings (NSString/NSMutableString), and I need to keep track of the original value; mostly because there are some Objective-C APIs that use object identity. The worst part is that fully initialising the PyUnicodeObject fields often isn’t necessary as a lot of Objective-C strings aren’t used as strings in Python code. The PyUnicodeObject (via its leading PyASCIIObject member) currently uses 7 flag bits including 2 for kind. Would it be worth adding an 8th big to indicate that string is a virtual subclass, and that the internals should not be touched directly? (This would require changing some of the macros; at the time of PEP 393 it Martin ruled YAGNI ... but is this something that might reasonably be reconsidered, if someone did the work. Which I am considering, but not committing to.) -jJ

4 4

between block and function [was: Custom Code Folding: Standardized Rules and Syntax?]
by Jim J. Jewett 19 Jul '17

19 Jul '17

There have been times when I wanted to group portions of a module, class, or function. Giving names to each of those groupings would be useful, and would be appropriate for python-level changes. That said, it would probably need more proof in the wild first, so the first step would be getting support in an editor (such as PyCharm), and the second would be an informational PEP on how to standardize the practice, like PEP 257 does for docstrings. https://www.python.org/dev/peps/pep-0257/ For these steps, you would probably have to use a comment or string convention, such as #{{{ or ":(group X ", though I suppose an external file (similar to the stub files for typing) is also possible. If these take off enough that the line noise gets annoying, that will prove the need, and getting support from python itself will be a lot easier. Example code :group C class C1: :group emulateFoo ... :group manageBar def xx(self, bar, name, val=None): :group setupBar ... :group actualXX class C2: ... :group D

2 1