Mailman 3 March 2011 - Python-ideas

mixins as decorators vs inheritance [was: Automatic comparisons by default]
by Jim Jewett 18 Mar '11

18 Mar '11

On Thu, Mar 17, 2011 at 9:48 AM, Mike Graham <mikegraham(a)gmail.com> wrote: > On Wed, Mar 16, 2011 at 5:48 PM, Masklinn <masklinn(a)masklinn.net> wrote: >> (also, since I apparently completely missed this, >> what was the rationale of making it a class >> decorator rather than, say, a mixin?) > Perhaps the better question is "Why do we ever > do mixins through inheritance?" That is a good question, and I was tempted to switch, until I realized that there are some good answers... (a) So we can do an isinstance check (b) So we can more easily override parts of the mixin -jJ

6 11

exec_closure (was Re: Assignment decorators)
by Eric Snow 17 Mar '11

17 Mar '11

During sprints I spent some time getting cozy with the python internals and the result is an exec_closure function I've put in an extension module. Essentially, it exposes the PyEval_EvalCodeEx function, so you can do execs with closures and arbitrary code objects, like Nick suggested. He more or less mentored me through this, my first foray into the Underlying C code, so thanks Nick! In the next week I intend on cleaning it up, putting it on PyPI, and writing a bit about it. One example I hope to make is how to simulate a def-from using a decorator. My hope is to add new, meaningful capability to python, so we'll see how it goes. -eric On Thu, Mar 10, 2011 at 7:31 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > On Thu, Mar 10, 2011 at 2:02 PM, Guido van Rossum <guido(a)python.org> > wrote: > > On Thu, Mar 10, 2011 at 11:41 AM, Paul Moore <p.f.moore(a)gmail.com> > wrote: > >> On 10 March 2011 12:49, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > >>> I actually agree it is a major weakness of the syntax. You can play > >>> games with "from" to rearrange the line. For example: > >>> > >>> def name from builder(param_spec): > >>> code_block > >>> > >>> as sugar for: > >>> > >>> name = builder("name", param_spec_obj, code_obj) > >> > >> Yes, I like that better... > > > > I'd like it better if it came with a default builder implementation > > that would create regular functions, so that > > > > def name(<params>): > > <block> > > > > was equivalent to > > > > def name from <defaultbuilder>(<params>): > > <block> > > Yeah, I was thinking that having builde equivalents for functions and > generators would be an interesting way to go. Builder objects may need > to be passed some extra parameters to make that feasible, though > (specifically, a reference to globals(), as well as the closure > fulfilment details). > > One interesting side effect of that is the ability to have a > never-yields generator without having to insert a dummy never-executed > yield statement anywhere. > > > But I don't see a reasonable way to do that. > > > > Also I think it's confusing to have both > > > > @some_magic > > def name(<params>): ... > > > > and > > > > def name from some_magic(<params): ... > > > > with different semantics. > > Don't forget: > > @more_magic > def name from some_magic(params): > code > > I'm *far* from convinced this is a good idea, but it at least meets > the holder of making something new possible. > > If exec() was enhanced to allow correct execution of closures and code > blocks expecting arguments, you could experiment with this using a > decorator and extracting the various pieces from a function object > after it had already been created. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > http://mail.python.org/mailman/listinfo/python-ideas >

1 0

Automatic comparisons by default
by Facundo Batista 17 Mar '11

17 Mar '11

Two very related proposals: 1. On "!=", if Python doesn't find __ne__, use "not __eq__()". 2. On "<=", if Python doesn't find __le__, use "__eq__() or __lt__()". The same for ">=", of course. Some considerations: - You can give decorators that fill the spaces, but I think that would be less error prone, more clean and less surprising with those rules (specially the first one). - If somebody wants for != not to be the opposite of ==, both methods can be defined. The same for __le__ and __ge__. - All that would be needed for comparisons (for those normal objects where those rules apply) would be __le__ and __eq__, which is what you need to replace __cmp__. - *I* don't know if this behaviour will break something (couldn't find a case). What do you think? Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

11 21

The Descriptor Protocol...
by Martin Chilvers 16 Mar '11

16 Mar '11

G'day! Please excuse me if I have missed something obvious, but I have a question about the implementation of the descriptor protocol, and more specifically about the arguments passed to the '__get__' and '__set__' methods. According to Raymond Hettinger's "How-To Guide for Descriptors" at:- http://users.rcn.com/python/download/Descriptor.htm#invoking-descriptors The *pseudo* implementation of __getattribute__ is as follows:- def __getattribute__(self, key): "Emulate type_getattro() in Objects/typeobject.c" v = object.__getattribute__(self, key) if hasattr(v, '__get__'): return v.__get__(None, self) return v As mentioned above, this is obviously only pseudo-Python, but it serves to illustrate my question which is, why isn't the 'key' argument passed through to the '__get__' (and similarly, '__set__') methods? In seems to me that:- 1) In terms of API design/information flow through '__getattribute__' it feels like we 'drop' the 'key' when we call call a descriptor's '__get__' method. In other words, if '__getattribute__' gets to use the key when working out what to return it also seems natural to give the descriptor the same information. 2) It makes the implementation of some descriptor based tools much uglier. Take for example a type specification tool, we might have something like:- class Person(object): name = Str() address = Str() where 'Str' is a descriptor. It would be nice to know in the '__get__' and '__set__' methods of 'Str' which attribute is being accessed. Of course, I can get around this by either:- a) using a metaclass to harvest the descriptors and set the attribute name. This is fine, but:- - it forces me to use a metaclass ;^) - it means that I can't share descriptors because they are bound to a particular attribute name which has obvious scaleability implications. b) make the developer duplicate the attribute name when constructing the descriptor:- class Person(object): name = Str('name') address = Str('address') which, well, just smells, and conflicts with step 3 of TDD ;^) Again, apologies if I've missed the obvious - I've trawled the usual places and not found this mentioned anywhere... Thanks folks! Martin

23 102

Re: [Python-ideas] [Python-Dev] User conversions in custom string.Formatter
by Eric Smith 16 Mar '11

16 Mar '11

On 3/15/2011 9:07 PM, Andrew Svetlov wrote: > Ooh. Sorry. Of course I used > s = Formatter().format("{0!q}, {1!t}", a, b) > > I don't want to put those specific conversions into standard - that > doesn't make sense as you said. > MyFormatter().format(...) was intended to extend standard formatting > mechanic and it works pretty well for me. > > But I can use only single-character as converter names. > PEP 3101 doesn't limit that explicitly, it only says: > - there are !s and !r (later added !a) converters > - a custom Formatter class can define additional conversion flags > > You cannot specify multi-char converter name because > string:Formatter.parse (and maybe string:Formatter.get_field) > cannot parse that form. You can get whatever behavior you want by overriding the .parse() method in addition to convert_field.

2 2

Re: [Python-ideas] [Python-Dev] User conversions in custom string.Formatter
by Andrew Svetlov 16 Mar '11

16 Mar '11

Well, let's discuss in python-ideas. I mean my custom formatter like: class Formatter(string.Formatter): def convert_field(self, value, conversion): if 'q' == conversion: return qname(value) elif 't' == conversion: return format_datetime(value) else: return super(Formatter, self).convert_field(value, conversion) I can use it like: s = "{0!q}, {1!t}".format(a, b) !q and !t are my extensions to formatter conversions. It's good enough but I prefer to see s = "{0!qname}, {1!time}".format(a, b) or something like that. It cannot break any existing rule (you still can use "{0!qname:<20}" for example) but you can use more readable names for converters. There are only changed file is Objects/stringlib/string_format.h as I can see (I don't count documentation and unitests). On Wed, Mar 16, 2011 at 2:25 AM, Eric Smith <eric(a)trueblade.com> wrote: > On 03/15/2011 08:07 PM, Andrew Svetlov wrote: >> >> As PEP 3101 http://www.python.org/dev/peps/pep-3101/ says (and current >> Python does) user can specify conversions like "{0!s}". >> In custom formatters (derived from string.Formatter) he can override >> convert_field method to add custom conversions. >> >> I experimented with that last month and found it very convenient. >>> >>> From my perspective custom conversions are very close to 'filters' >> >> from html template engines (jinja2, mako, django etc). >> While I like to see custom conversions simple and easy I don't wan't >> to bring 'cascading' and 'parametrization' to standard formatting. >> >> But why don't relax spec a bit and allow to use any word (with >> 'identifier' semantix, I mean str.isidentifier()) as conversion name? >> Proposed update doesn't make any backward incompatibility (both python >> and public C API are not changed), it's clean and obvious. >> And it's very funny to use words instead single characters for custom >> user-specific conversions. >> >> What do you think? > > This should be in python-ideas, but I don't know if Andrew is subscribed, so > I'll reply here until it gets out of control. > > Andrew: Could you give a code sample of what you'd like to see? > > Eric. > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com >

2 1

Re: [Python-ideas] Assignment decorators (Re: The Descriptor Protocol...)
by Ben Rudiak-Gould 10 Mar '11

10 Mar '11

(This will probably be unthreaded, because I don't have the original thread root in my mailbox and I didn't want to reply to a random deeply nested message.) All syntax aside, I'm unhappy with the semantics of this idea, and I'd like some clarification. Specifically, I'd like to know what a name is. First, namedtuple: >>> import collections >>> def f(): foo = collections.namedtuple('foo', 'a b') return foo, foo(a=1, b=2) >>> f() (<class '__main__.foo'>, foo(a=1, b=2)) >>> The namedtuple I defined is not __main__.foo; there's no such symbol. And it's not foo either, by the time it's asked for its repr(). The name argument to namedtuple has never been more than a hack that kinda-sorta works in the obvious cases. It's useful for debugging, not so much for production code. The same is true of class and function names, and they do have special syntax, but even so, this is not the sort of thing I'd want to enshrine in new syntax. If there's going to be new syntax, it should be part of a new general facility for runtime introspection of assignment targets, not a hacky way of passing a stringified identifier that has no meaning outside of the context of the caller. In Python right now, you can do this: def deftuple(name, fields): globals()[name] = collections.namedtuple(name, fields) deftuple('foo', 'a b') or this: def namedtupleclass(name, args, body): fields = body.pop('fields') module = body.pop('__module__') assert not args and not body return collections.namedtuple(name, fields) class foo(metaclass=namedtupleclass): fields = 'a b' I like these better than new syntax, because their level of hackiness roughly matches the intrinsic hackiness of the thing that they are doing. (By the way, I first wrote my metaclass to pass (module + '.' + name) as the first argument to namedtuple, but discovered that namedtuple doesn't support dotted tuple names. So much for that pathetic attempt at proper scoping.) overridable_property seems quite different, if I understand it correctly. In this case no string representation of the name is ever leaked to the outside world, and there's no problem of scope naming. Instead, the issue is that you want to create several related attributes that can refer to each other, from a common base name. It seems to me that the most straightforward way of doing this is: def def_overridable_property(name, classdict): classdict['get_' + name] = ... classdict['set_' + name] = ... classdict[name] = ... class Foo: def_overridable_property('foo', locals()) I realize that this is illegal by the current documentation of locals(), but there's no reason that it couldn't be legal and documented, is there? It might slow down the execution of class definitions; do those need to be fast? I wouldn't propose such a thing for function locals, but class locals get turned into a dictionary in the end anyway. Again, I like this solution because it seems roughly as hacky as the thing that it's designed to do. -- Ben

1 0

Literate python?
by Mike Meyer 10 Mar '11

10 Mar '11

Wild idea, swiped directly from haskell/ghc: How about making the python interpreter just a little bit smarter, to support literate programming? Add a 'literate python' mode, triggered by file type being .pyl, a '-l' option, or the interpreter being run as 'lpython', then the compiler does a little bit of filtering before compiling (and potentially saving .pyc/.pyo) the file. If the first non-white-space character after the shebang line (if present) is a backslash, then the compiler ignores lines until it sees a line consisting of \begin{code} (which could be the first line), then compiles lines until it sees a line consisting of \end{code}, after which it switches back to searching for \begin{code}. Otherwise, all lines (again, after the shebang line, if present) are treated as comments until the compiler sees a line starting with "> " (that's greater than followed by space) following an empty line, which causes the compiler to start stripping the "> " from lines and compiles them until it finds a line that doesn't start with "> ". <mike -- Mike Meyer <mwm(a)mired.org> http://www.mired.org/consulting.html Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

7 7

Bring back callable()
by Antoine Pitrou 07 Mar '11

07 Mar '11

Hello, Python 3 has removed callable() under the justification that's it's not very useful and duck typing (EAFP) should be used instead. However, it has since been felt by many people that it was an annoying loss; there are situations where you truly want to know whether something is a callable without actually calling it (for example when writing sophisticated decorators, or simply when you want to inform the user of an API misuse). The substitute of writing `isinstance(x, collections.Callable)` is not good, 1) because it's wordier 2) because collections is really not an intuitive place where to look for a Callable ABC. So, I would advocate bringing back the callable() builtin, which was easy to use, helpful and semantically sane. Regards Antoine.

17 21

str.split() oddness
by Mart Sõmermaa 07 Mar '11

07 Mar '11

IMHO, x.join(a).split(x) should be "idempotent" in regard to a. >>> foo = ['a', 'b', 'c'] >>> assert '|'.join(foo).split('|') == foo >>> foo = ['a'] >>> assert '|'.join(foo).split('|') == foo >>> foo = [] >>> assert ' '.join(foo).split() == foo And now the odd exception to the rule: >>> assert '|'.join(foo).split('|') == foo Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError That forces one to write special case code when using custom separators. Consider: # clean baz = dict(chunk.split('=') for chunk in baz.split()) # ugly baz = (dict(chunk.split('=') for chunk in baz.split("|")) if baz else {}) Our younger cousin Ruby has no such idiosyncrasies: >> foo = [] >> foo.join('|').split('|') == foo => true What is the reason for that oddity? Can we amend it? Best regards, Mart Sõmermaa

9 18

Python-ideas March 2011