Mailman 3 September 2014 - Python-ideas

Abstract metaclasses?
by Erik Bray 11 Sep '14

11 Sep '14

Hi all, I recently ran across an interesting (mis?)-feature at least in CPython that I couldn't find any specific justification for or against. The issue is that abstract *types*, while technically possible, don't behave as abstract classes. To exemplify, say I wanted to create a metaclass which itself has ABCMeta as a metaclass, and which has some abstract classmethod defined: >>> import abc >>> class Meta(type, metaclass=abc.ABCMeta): ... @abc.abstractmethod ... def foo(cls): pass ... Now for all intents and purposes Meta *is* an abstract type: >>> import inspect >>> inspect.isabstract(Meta) True >>> Meta.__abstractmethods__ frozenset({'foo'}) However, nothing prevents Meta from being used as a metaclass for another class, despite it being "abstract": >>> class A(metaclass=Meta): pass ... >>> A <class '__main__.A'> This is simply because the check for the Py_TYPFLAGS_IS_ABSTRACT flag is implemented in object_new, which is overridden by type_new for type subclasses. type_new does not perform this check. I'm perfectly fine if this is dismissed as too abstract or too academic to be useful, but I will mention that this came up in a real use case. The use case is in a hierarchy of metaclasses involved in a syntactic-sugary class factory framework involving creation of new classes via operators. So I just wonder if this is a bug that should be fixed, or at the very least a feature request. The IS_ABSTRACT flag check is cheap and easy to add to type_new, in principle. In the meantime a workaround, which doesn't seem too terrible, is simply to define something I called AbstractableType: >>> class AbstractableType(type): ... def __new__(mcls, name, bases, members): ... if inspect.isabstract(mcls): ... raise TypeError( ... "Can't instantiate abstract type {0} with " ... "abstract methods {1}".format( ... mcls.__name__, ', '.join(sorted(mcls.__abstractmethods__)))) ... return super(AbstractableType, mcls).__new__(mcls, name, bases, members) ... Now create an abstract metaclass with (meta-)metaclass abc.ABCMeta: >>> class AbstractMeta(AbstractableType, metaclass=abc.ABCMeta): ... @abc.abstractmethod ... def foo(cls): pass ... Creating a class with metaclass AbstractMeta fails as it should: >>> class A(metaclass=AbstractMeta): pass ... Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 7, in __new__ TypeError: Can't instantiate abstract type AbstractMeta with abstract methods foo However, AbstractMeta can be subclassed with a concrete implementation: >>> class ConcreteMeta(AbstractMeta): ... def foo(cls): print("Concrete method") ... >>> class A(metaclass=ConcreteMeta): pass ... >>> A.foo() Concrete method Thanks, Erik

2 5

Proposal: New syntax for OrderedDict, maybe built-in
by Norman Hooper 11 Sep '14

11 Sep '14

Hi there, I'd like to propose that the OrderedDict get a more readable syntax, the way the syntax for set changed from set(['spam', 'ham', 'eggs']) to {'spam', 'ham', 'eggs'} when it became a built-in type in Python 2.4. The way set is unordered like the keys of a dict, similarly list is ordered like the keys of an OrderedDict. In that sense, what {'ham', 'eggs'} is to {'ham': 'spam', 'eggs': 'spam'}, so ['ham', 'eggs'] is to ... my proposal of a cleaner OrderedDict syntax ... ['ham': 'spam', 'eggs': 'spam']. So a "dict" is like a "set" (hence curly braces) of key-value pairs. And an "OrderedDict" is like a "list" (hence square braces) of key-value pairs. (Of course, I am ignoring how lists can have duplicate items. An ordered set would carry the analogy further, but it wouldn't help illustrate my syntax proposal.) I find "['ham': 'spam', 'eggs': 'spam']"" unambiguous, and more readable than "OrderedDict([('ham', 'spam'), ('eggs', 'spam')])". I work with OrderedDict a lot, because JSON represents an OrderedDict and I need to work with JSON a lot. In an attempt to make the OrderedDicts that I work with more readable, I wrote a module to parse a string representation of an OrderedDict that uses my proposed syntax. You can find it here: https://github.com/kaapstorm/listdict With the ubiquity of JSON, it may also be time to promote OrderedDict to a built-in type too. Kind regards, Norman

5 6

Re: [Python-ideas] yield from __setitem__ and __setattr__
by Martin Teichmann 10 Sep '14

10 Sep '14

Hi Guido, Hi List > I think it's too soon to start inventing new syntax to address these; instead, I strongly > recommend changing your APIs to use explicit method calls instead of the special > syntax you so favor. Even __getitem__() returning a Future feels awkward to me > (unless it's clear that the container you have in front of you is a container full of Futures). > [...] > But again I would like to establish asyncio as the de-factor standard for asynchronous work > before trying to tweak the language more I agree that first we need to settle a bit and look how everything works, but hey, this is python-ideas, the list for "discussing speculative language ideas". Limiting all APIs to just use explicit method calls unfortunately leads to a very Java-like language that misses all the cool features of python. And this is not limited only to low-level stuff, but also high level interfaces. Right now I am actually working on a distributed framework, where users which are not professional programmers are supposed to write their own plugins. My boss is constantly bugging me "all that yield from stuff looks complicated, can't you just put it in a function the user calls so they don't see it anymore?" Well, as you know, I cannot. Even in the highest level of abstraction, every function that needs to do I/O has to be yielded from. Imagine the client side of a Big Data service like google earth, what syntax do you prefer, "yield from map.get_section(long1, long2, long_step, lat1, lat2, lat_step)", or simply "yield from map[long1:long2:long_step, lat1:lat2:lat_step]"? Until now, a yield from was just a rather rare case used only on special occasions. But once asyncio becomes more wide spread, it will simply be everywhere. It will be so ubiquitous that I even thought it should not be spelled out anymore at all - every expression evaluating to a coroutine should automatically be yielded from. But that would actually kill the advantange of this construct - the cooperative multitasking we are doing with yield from. The programmer of a piece of code would not know anymore where other tasks step in, so all the problems with race conditions re-appear. Given that it will be used very often in the future, I thought we could even use a special character to represent the yield from, say, a $. Then code like the following would be possible, please don't hate me for the Perl-look: for a, $b, c in some_iterator(): $ob.foo = $calculate(a + b, c) which would be equivalent to for a, _b, c in some_iterator(): b = yield from _b yield from ob.__setattr__("foo", yield from calculate(a + b, c)) But I guess with that I am just way too speculative... Greetings Martin

2 2

yield from __setitem__ and __setattr__
by Martin Teichmann 09 Sep '14

09 Sep '14

Hi list, I am a happy user of yield from, especially while also using asyncio. A great feature is that one can seamlessly convert synchronous code into asynchronous code by adding the appropriate yield froms and coroutine statements at the proper place, including special methods like __getitem__, as in "yield from something[7]". But we cannot yield from the special methods __setattr__ and __setitem__. That is unfortunate, as I am programming a proxy object for something remote over the network, and also __setitem__ might take some time. So I got the idea that yield from might also be used on the left side of an assigment, as in yield from o[3], yield from o.something = "a", "b" which should be equivalent to yield from o.__setitem__(3, "a") yield from o.__setattr__("something", "b") I know that with this proposal I am probably far up on the list of most ugly syntax proposed ever, so maybe someone has a better idea. As a side note: with asyncio I am using yield from a lot, and I think it is a pity that it is so long an clumsy a keyword-couple. Couln't it be shortened? I'm thinking about just using "from", especially in an asyncio context, "from sleep(3)" sounds bad to me, but still better than "yield from sleep(3)". Or maybe we could have an auto-yielder, by just declaring that every expression that evaluates to a coroutine is automatically yielded from, as long as the calling function is a coroutine itself. Greetings Martin

2 1

Have ConfigParser handle sectionless config files by default
by Kernc 08 Sep '14

08 Sep '14

Hi all, I recently opened this bug report [1] stating that ConfigParser not handling configuration files without sections (or those that have options above first section definition) is an issue. ConfigParser* should be able to parse "POSIX" config files (key=value, no sections). I'd like to lobby even for it being the default behavior without requiring a strict=False argument. If interested, please have a look at the bug report for some further arguments. Thanks, [1]: http://bugs.python.org/issue22253 * It's called ConfigParser, not IniParser.

1 0

My apologies
by Norman Hooper 04 Sep '14

04 Sep '14

I completely missed the same proposal made by Anthony Towns in March this year. Kind regards Norman

1 0

Subprocess: Add an encoding argument
by Paul Moore 03 Sep '14

03 Sep '14

At the moment, subprocess offers two options for handlingthe standard IO streams of the child. By default, the streams are binary, or you can set universal_newlines to get text-mode streams with universal newline handling enabled. With universal_newlines, the encoding of the streams is the default value for the environment (whatever locale.getpreferredencoding() returns). However, there can be cases where you want finer control over the encoding to use (for example, if you run Python in a subprocess and set PYTHONIOENCODING). I propose adding an "encoding" parameter to subprocess.Popen (and the various wrapper routines) to allow specifying the actual encoding to use. Obviously, you can simply wrap the binary streams yourself - the main use for this facility would be in the higher level functions like check_output and communicate. Does this seem like a reasonable suggestion? Paul

8 20