On Fri, Feb 21, 2014 at 1:37 PM, Terry Reedy <tjreedy@udel.edu> wrote:

Off the top of my head, I cannot think of any methods that return self. (If there are some, someone please remind or inform me.)

That's because I don't like this pattern. :-) But there's some code in the stdlib that uses it (IIRC Eric Raymond was a fan), and I'm sure there's lots of 3rd party Python code too. -- --Guido van Rossum (python.org/~guido)

On 2014-02-21, at 23:00 , spir <denis.spir@gmail.com> wrote:

Also, I don't find the idea of having a builtin construct for such hacks a good idea. Libs for which this may be practicle can return self --end of the story.

That has two issues though: 1. it makes chainability a decision of the library author, the library user gets to have no preference. This means e.g. you can't create a tree of elements in ElementTree in a single expression (AFAIK Element does not take children parameters). With cascading, the user can "chain" a library whose author did not choose to support chaining (in fact with cascading no author would ever need to support chaining again). 2. where a return value can make sense (and be useful) the author *must* make a choice. No way to chain `dict.pop()` since it returns the popped value, even if `pop` was only used for its removal-with-shut-up properties. With cascading the user can have his cake and eat it: he gets the return value if he wants it, and can keep "chaining" if he does not care.

On Sat, Feb 22, 2014 at 9:43 AM, Masklinn <masklinn@masklinn.net> wrote:

On 2014-02-21, at 23:00 , spir <denis.spir@gmail.com> wrote:

...

Also, I don't find the idea of having a builtin construct for such hacks a good idea. Libs for which this may be practicle can return self --end of the story.

That has two issues though:

1. it makes chainability a decision of the library author, the library user gets to have no preference. This means e.g. you can't create a tree of elements in ElementTree in a single expression (AFAIK Element does not take children parameters). With cascading, the user can "chain" a library whose author did not choose to support chaining (in fact with cascading no author would ever need to support chaining again).

Right. That's the main point behind this: it gives the *caller* the choice of whether to chain or not. That's really the whole benefit, right there. ChrisA

On 2014-02-22, at 11:23 , spir <denis.spir@gmail.com> wrote:

...

2. where a return value can make sense (and be useful) the author *must* make a choice. No way to chain `dict.pop()` since it returns the popped value, even if `pop` was only used for its removal-with-shut-up properties. With cascading the user can have his cake and eat it: he gets the return value if he wants it, and can keep "chaining" if he does not care.

... not there (if I understand you well; not quite 100% sure). In fact, I find this point rather a counter-argument, something to avoid (again, if I understand). What I mean is that executing given methods should have consistent effect

Executing the method always has the same effect on its subject. That it may not be used for the same purpose is a different issue and common: a[k] = v can be used to add a new (k, v) pair or to update a key to a new value (in fact Erlang's new map construct makes the difference and provides for an update-only version). Even more so for values returned by mutating methods, as far as I no there is no rule that they must be used if the method is only called for its side effects.

also, you should use the right method for the right task: if you don't want a stack's top item _and_ have it removed, then don't use 'pop', otherwise you are misleading readers (including yourself maybe, later) (note 'pop' is just a convenience utility for this very case; we could just read and remove in 2 steps).

Not only is `dict.pop` an expression (which del is not, and thus del can't be used in some contexts where pop would be useable, e.g. in a lambda) but dict.pop can also handle a non-existent value at the key, doing so with `del` requires adding a supplementary conditional. dict.pop's convenience oft makes it the right method for the right task, even if the case is "remove the key a from the dict if it's there". But if you don't like the pop example there are others. The sibling thread "Joining dicts again" for instance: with cascading it's a non-problem, you can just cascade `update` calls on a base dict and you get a fully correct (as opposed to dict(a, **b) which is only correct if all of b's keys are strings) single-expression union.

On Sun, Feb 23, 2014 at 2:44 AM, Alan Cristhian Ruiz <alan.cristh@gmail.com> wrote:

What is wrong with the current sintax?:

'abcd'\ .upper()\ .lower()\ .title()

It doesn't have each method operate on the original object. It's not easy to see with strings, but try this: list_of_numbers = [1,2] list_of_numbers.append(3) list_of_numbers.append(4) list_of_numbers.append(5) Now write that without repeating list_of_numbers all the way down the line. ChrisA

On Sun, Feb 23, 2014 at 1:25 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

it's a fundamentally bad idea to blur that distinction: you should be able to tell *at a glance* whether an operation is mutating an existing object or creating a new one...

Compare:

seq = get_data() seq.sort()

seq = sorted(get_data())

Now, compare that with the proposed syntax as applied to the first operation:

seq = []->extend(get_data())->sort()

That *looks* like it should be a data transformation pipeline, but it's not - each step in the chain is mutating the original object, rather than creating a new one. That's a critical *problem* with the idea, not a desirable feature.

Except that it doesn't. The idea of using a different operator is that it should clearly be mutating the original object. It really IS obvious, at a glance, that it's going to be returning the existing object, because that operator means it will always be. I believe that naming things that don't matter is a bad idea. We don't write code like this: five = 5 two = 2 print("ten is",five*two) because the intermediate values are completely insignificant. It's much better to leave them unnamed. (Okay, they're trivial here, but suppose those were function calls.) In a GTK-based layout, you'll end up creating a whole lot of invisible widgets whose sole purpose is to control the layout of other widgets. In a complex window, you might easily have dozens of those. (Same happens in Tkinter, from what I gather, but I haven't much looked into that.) Naming those widgets doesn't improve readability - in fact, it damages it, because you're left wondering which insignificant box is which. Leaving them unnamed and just part of a single expression emphasizes their insignificance. ChrisA

The other means you're doing declarative stuff, transforming objects into new temporary objects. That distinction is huge

I guess that's where people disagree. I think the distinction is not huge. Whether imperatively constructing something or "declaratively" (???) doing transformations, the *meaning* of the code is the same: start from some *foo* and do *some stuff *on *foo *in sequence until the *foo *is what I want. Whether it's implemented using mutation or allocation/garbage-collection is an implementation detail that clouds our view of a higher-level semantic: initializing an object with some stuff. In fact, this distinction is so meaningless that many languages/runtimes will turn one into the other as an optimization, because the semantics are exactly the same. On Sat, Feb 22, 2014 at 9:06 PM, Andrew Barnert <abarnert@yahoo.com> wrote:

On Feb 22, 2014, at 18:48, Chris Angelico <rosuav@gmail.com> wrote:

...

On Sun, Feb 23, 2014 at 1:25 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

...

That *looks* like it should be a data transformation pipeline, but it's not - each step in the chain is mutating the original object, rather than creating a new one. That's a critical *problem* with the idea, not a desirable feature.

Except that it doesn't. The idea of using a different operator is that it should clearly be mutating the original object. It really IS obvious, at a glance, that it's going to be returning the existing object, because that operator means it will always be.

The difference between the look of nested statements and giant expressions in Python is much larger than the difference between the look of . and ->. One structure means you're doing imperative stuff, mutating one value on each line. The other means you're doing declarative stuff, transforming objects into new temporary objects. That distinction is huge, and the fact that it's immediately visible in Python i's one of the strengths of Python over most other "multi-paradigm" languages.

...

I believe that naming things that don't matter is a bad idea. We don't write code like this:

five = 5 two = 2 print("ten is",five*two)

But in real life code, this would be something like rows * columns, and even if rows and columns are constant, they're constants you might want to change in a future version of the code, so you _would_ name them.

And if, as you say, they're actually function calls, not constants, I think most people would write:

rows = consoleobj.getparam('ROWS') cols = consoleobj.getparam('COLS') cells = rows * cols

... rather than try to cram it all in one line.

...

because the intermediate values are completely insignificant. It's much better to leave them unnamed. (Okay, they're trivial here, but suppose those were function calls.) In a GTK-based layout, you'll end up creating a whole lot of invisible widgets whose sole purpose is to control the layout of other widgets. In a complex window, you might easily have dozens of those. (Same happens in Tkinter, from what I gather, but I haven't much looked into that.) Naming those widgets doesn't improve readability - in fact, it damages it, because you're left wondering which insignificant box is which. Leaving them unnamed and just part of a single expression emphasizes their insignificance.

All you're arguing here is that PyGtk is badly designed, or that Gtk is not a good match for Python, so you have to write wrappers. There's no reason the wrapper has to be fluent instead of declarative. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Feb 23, 2014, at 3:51, spir <denis.spir@gmail.com> wrote:

The 2 methods above are not semantically equivalent. In the mutative case, you're talking about one single *thing* (the representation of one thing in the model, correctly "reified" in code). This thing may be multiply ref'ed, because it is a thing and as such has multiple *aspects* or *roles* (or competences?). In the functional case, you making up new objects at every step, thus if there were ref's they would be broken. We should only use such a style for non-things, meaning for plain data (information) *about* things.

(This style is not appropriate for GUI widgets, which conceptually are things, and pretty often ref'ed. Instead widgets should be created on one go.)

But put that together with your other reply:

That's what I was about to argue. I don't understand why the python wrapper does not let you construct widgets in one go, with all their "equipment", since it's trivial and standard style in python (except, certainly, for adding sub-widgets to containers, as the sub-widgets exist and need to defnied by themselves).

The cleanest way to do this would then be to build the initializer(s) declaratively, then initialize the (mutable) object in one go, right? As for why PyGtk doesn't work that way, there are two reasons. First, it's deliberately intended to make Python Gtk code, C Gtk code, Vala Gtk code, etc. look as similar as possible. Gtk has a language-agnostic idiom that takes precedence over each language's idioms. This means you only have to write examples, detailed docs, etc. once, instead of once for every language with bindings. Second, from a practical point of view, it allow PyGtk to be a very thin wrapper around Gtk. And if you want to know why Gtk itself wasn't designed to be more Pythonic... Well, it wasn't designed for Python. It was originally designed for C, and then updated for C and Vala. So it has C/Vala-focused idioms. That's the same reason Qt and Wx have C++ idioms, Tk has Tcl idioms, WinForms has C#/VB idioms, Cocoa has ObjC idioms, etc. Unfortunately, none of the major GUI libraries was designed primarily with Python in mind, so we have to adapt.

On Feb 23, 2014, at 16:36, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:

Andrew Barnert wrote:

...

Unfortunately, none of the major GUI libraries was designed primarily with Python in mind, so we have to adapt.

Yes, but the adaptation can be in the form of wrappers that make the API more Pythonic. It shouldn't mean warping Python to make it fit the ways of other languages.

But remember that there is an advantage to Gtk, Qt, etc. having their own language-agnostic idioms. They have a hard enough time documenting the whole thing as it is; if they had to write completely different documentation for C, C++, Vala, Python, .NET, etc., we just wouldn't get any documentation. Of course there's also a disadvantage. PyGtk code doesn't look very Pythonic. I think the suggestions in this thread for a language change that allows people to write code that looks like _ neither_ Python _nor_ Gtk is a bad solution to the problem. But there is a real problem, and I understand why people are trying to solve it. So what is the solution? Maybe the best thing people can put their effort into is a higher-level, more Pythonic wrapper around the most painful parts of the PyGtk wrapper (like initialization)?

On Feb 23, 2014, at 23:00, Chris Angelico <rosuav@gmail.com> wrote:

On Mon, Feb 24, 2014 at 5:49 PM, Andrew Barnert <abarnert@yahoo.com> wrote:

...

But remember that there is an advantage to Gtk, Qt, etc. having their own language-agnostic idioms. They have a hard enough time documenting the whole thing as it is; if they had to write completely different documentation for C, C++, Vala, Python, .NET, etc., we just wouldn't get any documentation.

Point to note: When I'm trying to pin down an issue that relates to GTK on Windows, I'll sometimes switch between Pike and Python, since my installations of them embed different GTK versions. (I discovered a GTK bug that way.) Being able to translate my code from one language to another is extremely useful. And when I'm looking for docs and (especially) examples, it's really common to get them for some completely different language - it's easier to translate back from (say) Perl than to hunt down an example in the language I'm actually using.

A more Pythonic wrapper around object creation would pretty much look like what I was saying, except that it would take specific coding work. Actually, probably all it'd take is a module that imports all the PyGTK classes and wraps them in Steven's chain() function.

But that would be _less_ Pythonic, not more. The fact that mutating methods return None--and, more generally, the strong divide between mutation and transformation and between statements and expressions--is one of the major ways in which Python is idiomatically different from languages that are specifically meant to be fluent (like Smalltalk or C#) or that just never considered the design issue (like perl or JavaScript). You don't have to agree with Guido that method chaining is bad. But he's designed his language and stdlib to discourage it, and therefore anything you do in the opposite direction is fighting against the grain of the language and its standard idioms.

spir writes:

On 02/23/2014 06:06 AM, Andrew Barnert wrote:

...

All you're arguing here is that PyGtk is badly designed, or that Gtk is not a good match for Python, so you have to write wrappers. There's no reason the wrapper has to be fluent instead of declarative.

That's what I was about to argue. I don't understand why the python wrapper does not let you construct widgets in one go,

*Because* it's a *wrapper*, which leverages the gobject-introspection FFI. (gobject-introspection is an export-oriented FFI, rather than an import-oriented FFI like ctypes.)

with all their "equipment", since it's trivial and standard style in python (except, certainly, for adding sub-widgets to containers, as the sub-widgets exist and need to defnied by themselves).

I don't think you need to except subwidgets. They could be defined recursively by including calls to their constructors in the "description" of the parent. It might be tricky to find a pleasant way to express placement in containers with flexible placement disciplines, but for single-child, column, row, and grid widgets I don't see a problem at all. Now, that would be nice for initialization, but the GTK+ v3 API is very dynamic and insanely complicated. I think it would take a huge amount of work to do this at all well, and I suspect that your program would benefit only from beautification of initialization -- everything else would still look the same. (Of course many programs don't need a dynamic UI; I suppose that would be a benefit.) In any case, I don't find a "chained" API for something like GTK any more attractive than the repetition of object being mutated. They're both quite ugly, an accurate reflection of the underlying library which tried to be a better Xt, but ended up equally messy, compounded by being a lot bigger (and more poorly documented).

On Tue, Feb 25, 2014 at 12:10 AM, Andrew Barnert <abarnert@yahoo.com> wrote:

So I put together examples for both the PyGtk example that started this thread and the Java example that started the whole fluent-interface fad, along with an explanation of why method chaining is neither necessary in, nor a good fit for, Python even though it's very useful in languages like Java. See https://stupid-python-ideas.runkite.com/fluent-pythonic/ for the whole thing.

While I do think a method chaining operator would solve the problem generically, rather than requiring every module to do it individually, I do rather like your proposed window creation syntax. The thing is, Pike GTK is *almost* there: instead of creating a window and setting its title and border, you can create a window and pass it a mapping (dict) specifying the title and border. (PyGTK doesn't have anything of the sort, it seems. I haven't looked into PyGObject, which is supposed to be the new great thing; it might have that.) But it's not quite all the way, because you can't stuff children into them. Being able to do the whole job in the constructor is extremely tempting. It'd work beautifully for the objects where you basically just call add() with each thing (just provide a list of children to be added), but not so well when you want to specify parameters (eg specifying how spare space should be allocated). I'm not sure how that ought to be done. I could come up with something where you pass it a list of tuples, but I'm not sure that completely covers the issue either. There's no perfect solution, which is why the search continues. I freely admit that the suggestion I made at the beginning of this thread is unideal; it's un-Pythonic, it's a heavy language change, and it'd take a huge amount of justification to go anywhere. But there is a problem still, that it's trying to solve. ChrisA

I agree with nick, this looks like a transformation chain. Each step

On Feb 23, 2014 10:25 AM, "Ron Adam" <ron3200@gmail.com> wrote: transforming the "new" result of the previous step.

seq = []->extend(get_data())->sort()

To make it pythonic ...

The operator you want is one for an in place method call. If we apply the

"+=" pattern for the '__iadd__' method call syntax, to the more the general '.' method syntax, we get ".=", the in place method call syntax.

seq = [] seq .= extend(get_data()) # In place method call.

In place method calls seem quite reasonable to me.

And then to get the rest of the way there, allow chained "in place"

method calls.

seq = [] .= extend(get_data()) .= sort()

Which should be a separate pep from the ".=" enhancement.

BTW... allowing ".=" could mean a class could have one __iget_method__

attribute instead of multiple __ixxxx___ methods. (Or something like that.)

Cheers, Ron

I like this syntax. It easy to tell what exactly is getting mutated.

On 02/23/2014 05:52 PM, Stephen J. Turnbull wrote:

Ron Adam writes:

...

The operator you want is one for an in place method call.

...

seq = [] .= extend(get_data()) .= sort()

That looks like anything but Python to me.

Is it really all that different from this?

...

...

"Py" . __add__("th") . __add__("on") 'Python'

The '.=' just says more explicitly that self will be returned after the method call. It wouldn't alter the string example here, since self isn't returned. But for mutable objects, it's an explicit reminder that it mutates rather than returns a new object. In the case that there is no __iget_method__ method, it would give an error. So it's not a make everything into a chain tool.

If I really thought of that as a single operation, I'd do something like

class ScarfNSort(list): def __init__(self): self.extend(get_data()) self.sort()

seq = ScarfNSort()

If it doesn't deserve a class definition, then the repeated references to 'seq' wouldn't bother me.

N.B. '.=' shouldn't be called "in-place": 'sort' and 'extend' are already in-place. The word would be "chain," "cascade," or similar.

It's a chain only if you link more than one in sequence. Cheers, Ron

On 02/24/2014 01:34 AM, Andrew Barnert wrote:

...

...

On 02/23/2014 05:52 PM, Stephen J. Turnbull wrote:

...

...

Ron Adam writes:

...

The operator you want is one for an in place method call.

...

seq = [] .= extend(get_data()) .= sort()

That looks like anything but Python to me.

Is it really all that different from this?

...

...

...

>>>"Py" . __add__("th") . __add__("on") 'Python' Well, yes, it is. But, more importantly, who cares? That code is horribly unreadable and unpythonic. OK, I don't think PEP8 has a guideline saying "don't call __add__ when you can just use +", but only because it's so obvious it doesn't need to be stated. (And I'm pretty sure it_does_ have a guideline saying not to put spaces around the attribute dot.)

So, is your argument is "my code looks kind of like some horribly unreadable and unpythonic, but legal, code, and therefore it should also be legal despite being unreadable and unpythonic?"

Wow, tough crowd here.. :-) Both the separation of the '.', and the use of the already special __add__ isn't important as far as the actual suggestion is concerned. Those are unrelated style issues. You would probably see it used more often like this... def names(defaults, pos_names, pos_args, kwds): return {}.=update(defaults) \ .=update(zip(pos_names, pos_args) \ .=update(kwds) Normally .update returns None. The reason for that is so that it's clear you are mutating an object instead of creating a new one. By using .=, it can return self, but still maintain the clarity between mutation and non-mutation. This particular syntax is consistent with the use of OP+equal to mean mutate in place. But you might prefer, "..", or something else. The other alternative is to use a function. But it would be difficult to get the same behaviour along with the same efficiency. Regards, Ron

On 02/24/2014 09:12 AM, Paul Moore wrote:

On 24 February 2014 15:01, Ron Adam<ron3200@gmail.com> wrote:

...

...

You would probably see it used more often like this...

def names(defaults, pos_names, pos_args, kwds): return {}.=update(defaults) \ .=update(zip(pos_names, pos_args) \ .=update(kwds) How is this better than

def names(defaults, pos_names, pos_args, kwds): ret = {} ret.update(defaults) ret.update(zip(pos_names, pos_args) ret.update(kwds) return ret

(I originally named the return value _ to cater for the tendency to insist on punctuation rather than names in this thread, but honestly, why*not* name the thing "ret"?)

I get the idea of chained updates, I really do. But translating between mutation of a named value and chained updates is pretty trivial, so I don't see how this is anything but a case of "follow the preferred style for the language/API you're using". And Python uses updating named values, why is that so bad?

It's not bad, just not as good. The chained expression is more efficient and can be used in places where you can't use more than a single expression. The point is to maintain both a visual and computational separation of mutable and immutable expressions. Compare the byte code from these. You can see how the chained version would be more efficient. Cheers, Ron def names(defaults, pos_names, pos_args, kwds): ret = {} ret.update(defaults) ret.update(zip(pos_names, pos_args)) ret.update(kwds) return ret

...

...

dis(names) 2 0 BUILD_MAP 0 3 STORE_FAST 4 (ret)

3 6 LOAD_FAST 4 (ret) 9 LOAD_ATTR 0 (update) 12 LOAD_FAST 0 (defaults) 15 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 18 POP_TOP 4 19 LOAD_FAST 4 (ret) 22 LOAD_ATTR 0 (update) 25 LOAD_GLOBAL 1 (zip) 28 LOAD_FAST 1 (pos_names) 31 LOAD_FAST 2 (pos_args) 34 CALL_FUNCTION 2 (2 positional, 0 keyword pair) 37 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 40 POP_TOP 5 41 LOAD_FAST 4 (ret) 44 LOAD_ATTR 0 (update) 47 LOAD_FAST 3 (kwds) 50 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 53 POP_TOP 6 54 LOAD_FAST 4 (ret) 57 RETURN_VALUE By using the '.' we can see the difference. The byte code should be very close to this, even though this function will give an error if you try to run it. (Can't update None.) The actual difference would probably be replacing LOAD_ATTR with LOAD_MUTATE_ATTR, Which would call __getmutatemethod__ instead of __getmethod__. (or something similar to that, depending on how it's implemented.) def names(defaults, pos_names, pos_args, kwds): return {}.update(defaults) \ .update(zip(pos_names, pos_args)) \ .update(kwds)

...

...

dis(names) 2 0 BUILD_MAP 0 3 LOAD_ATTR 0 (update) 6 LOAD_FAST 0 (defaults) 9 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 12 LOAD_ATTR 0 (update)

3 15 LOAD_GLOBAL 1 (zip) 18 LOAD_FAST 1 (pos_names) 21 LOAD_FAST 2 (pos_args) 24 CALL_FUNCTION 2 (2 positional, 0 keyword pair) 27 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 30 LOAD_ATTR 0 (update) 4 33 LOAD_FAST 3 (kwds) 36 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 39 RETURN_VALUE

On 2014-02-24, at 17:08 , Ron Adam <ron3200@gmail.com> wrote:

On 02/24/2014 09:12 AM, Paul Moore wrote:

...

On 24 February 2014 15:01, Ron Adam<ron3200@gmail.com> wrote:

...

...

You would probably see it used more often like this...

def names(defaults, pos_names, pos_args, kwds): return {}.=update(defaults) \ .=update(zip(pos_names, pos_args) \ .=update(kwds) How is this better than

def names(defaults, pos_names, pos_args, kwds): ret = {} ret.update(defaults) ret.update(zip(pos_names, pos_args) ret.update(kwds) return ret

(I originally named the return value _ to cater for the tendency to insist on punctuation rather than names in this thread, but honestly, why*not* name the thing "ret"?)

I get the idea of chained updates, I really do. But translating between mutation of a named value and chained updates is pretty trivial, so I don't see how this is anything but a case of "follow the preferred style for the language/API you're using". And Python uses updating named values, why is that so bad?

It's not bad, just not as good. The chained expression is more efficient and can be used in places where you can't use more than a single expression.

The point is to maintain both a visual and computational separation of mutable and immutable expressions.

Compare the byte code from these. You can see how the chained version would be more efficient.

The chained version is not intrinsically more efficient, the Python compiler could be smart enough to not LOAD_FAST (ret) repeatedly (if that proves beneficial to execution speed, which I'm not even certain of, and either way it's going to be extremely minor compared to the actual cost of executing methods). AFAIK the peephole optimiser does not even bother eliding out pairs of STORE_FAST $name LOAD_FAST $name e.g. as far as I know a = foo() a.bar() compiles to: 0 LOAD_* 0 (foo) 3 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 6 STORE_FAST 0 (a) 9 LOAD_FAST 0 (a) 12 LOAD_ATTR 1 (bar) 15 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 18 POP_TOP the pair (6, 9) is a noop and could trivially be removed (in the absence of jumps around). According to [0] a patch implementing this (although without taking care of jumps) was rejected:

because apparently the additional six lines of code didn’t buy enough of a speed improvement for an uncommon case.

(although no link to the patch so he might have been optimizing the triplet of (STORE_FAST, LOAD_FAST, RETURN_VALUE)). If removing 2 bytecode instructions once in a while does not sway the core team, I really can't see removing a single one even more rarely doing so.

By using the '.' we can see the difference. The byte code should be very close to this, even though this function will give an error if you try to run it. (Can't update None.) The actual difference would probably be replacing LOAD_ATTR with LOAD_MUTATE_ATTR, Which would call __getmutatemethod__ instead of __getmethod__. (or something similar to that, depending on how it's implemented.)

Why? There's no need for LOAD_MUTATE_ATTR. And LOAD_ATTR calls __getattribute__ (and __getattr__ if necessary), a bound method is a form callable attribute, the bytecode for a method call (assuming an object on the stack) is LOAD_ATTR $attrname CALL_FUNCTION that the function mutates the original object (or not) has no relevance to attribute loading.

def names(defaults, pos_names, pos_args, kwds): return {}.update(defaults) \ .update(zip(pos_names, pos_args)) \ .update(kwds)

...

...

...

dis(names) 2 0 BUILD_MAP 0 3 LOAD_ATTR 0 (update) 6 LOAD_FAST 0 (defaults) 9 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 12 LOAD_ATTR 0 (update)

3 15 LOAD_GLOBAL 1 (zip) 18 LOAD_FAST 1 (pos_names) 21 LOAD_FAST 2 (pos_args) 24 CALL_FUNCTION 2 (2 positional, 0 keyword pair) 27 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 30 LOAD_ATTR 0 (update)

4 33 LOAD_FAST 3 (kwds) 36 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 39 RETURN_VALUE

That bytecode's not correct for the case: * the return value of each method call needs to be discarded with a POP_TOP * LOAD_ATTR needs an object on the stack so you need a DUP_TOP before each LOAD_ATTR (update) (you can create the correct bytecode with something like byteplay, it'll work) [0] http://www.coactivate.org/projects/topp-engineering/blog/2008/11/03/optimizi...

On 02/24/2014 01:59 PM, Masklinn wrote:

...

...

By using the '.' we can see the difference. The byte code should be very close to this, even though this function will give an error if you try to run it. (Can't update None.) The actual difference would probably be replacing LOAD_ATTR with LOAD_MUTATE_ATTR, Which would call __getmutatemethod__ instead of __getmethod__. (or something similar to that, depending on how it's implemented.) Why? There's no need for LOAD_MUTATE_ATTR. And LOAD_ATTR calls __getattribute__ (and __getattr__ if necessary), a bound method is a form callable attribute, the bytecode for a method call (assuming an object on the stack) is

LOAD_ATTR $attrname CALL_FUNCTION

that the function mutates the original object (or not) has no relevance to attribute loading.

Turn it around... If an object doesn't have a mutate-attribute-loader.. Then you will get an error before the CALL_FUNCION instead of during it or after it, without making any changes to existing method/function call code paths.

...

def names(defaults, pos_names, pos_args, kwds): return {}.update(defaults) \ .update(zip(pos_names, pos_args)) \ .update(kwds)

...

...

...

...

>> dis(names) 2 0 BUILD_MAP 0 3 LOAD_ATTR 0 (update) 6 LOAD_FAST 0 (defaults) 9 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 12 LOAD_ATTR 0 (update)

3 15 LOAD_GLOBAL 1 (zip) 18 LOAD_FAST 1 (pos_names) 21 LOAD_FAST 2 (pos_args) 24 CALL_FUNCTION 2 (2 positional, 0 keyword pair) 27 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 30 LOAD_ATTR 0 (update)

4 33 LOAD_FAST 3 (kwds) 36 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 39 RETURN_VALUE

That bytecode's not correct for the case:

* the return value of each method call needs to be discarded with a POP_TOP

* LOAD_ATTR needs an object on the stack so you need a DUP_TOP before each LOAD_ATTR (update) (you can create the correct bytecode with something like byteplay, it'll work)

Actually, for it to work the way I was thinking, it needs a matched pair to replace LOAD_ATTR and CALL_FUNCTION. The alternate call function bytecode would leave the funciton on the stack and give an error if the returned anything other than None. But I think that's too many new Byte codes. Even one new byte code is a hard sell. The idea is really just a more limited version of cascading with an error for non-mutatables used with that syntax, and am error if any value is returned when using with that syntax. So if you see this particular syntax you will know instantly the intent is to mutate the subject. It might be a fun patch to play with and/or try to do. But I don't think it would ever get approved. The alternative is to use the existing byte codes as you describe and that removes any (however small they might be) performance benifits. Cheers, Ron

On Tue, Feb 25, 2014 at 2:16 AM, Steven D'Aprano <steve@pearwood.info> wrote:

On Mon, Feb 24, 2014 at 09:01:07AM -0600, Ron Adam wrote:

...

You would probably see it used more often like this...

def names(defaults, pos_names, pos_args, kwds): return {}.=update(defaults) \ .=update(zip(pos_names, pos_args) \ .=update(kwds)

Normally .update returns None. The reason for that is so that it's clear you are mutating an object instead of creating a new one.

By using .=, it can return self, but still maintain the clarity between mutation and non-mutation.

How does the update method know whether it is being called via . or via .= ? I'm trying to understand how you think this is supposed to work, and not having much success. Can you give a sketch of how this .= thingy is supposed to operate?

I don't know how his plan is, but mine was for the function to continue to return None, or 42, or "spam", or whatever it likes, and for that to be ignored. The expression result would be the initial object, and the actual function return value is discarded. Function doesn't need any rewriting. ChrisA

On Feb 24, 2014, at 7:01, Ron Adam <ron3200@gmail.com> wrote:

On 02/24/2014 01:34 AM, Andrew Barnert wrote:

...

So, is your argument is "my code looks kind of like some horribly unreadable and unpythonic, but legal, code, and therefore it should also be legal despite being unreadable and unpythonic?"

Wow, tough crowd here.. :-)

Both the separation of the '.', and the use of the already special __add__ isn't important as far as the actual suggestion is concerned. Those are unrelated style issues.

You would probably see it used more often like this...

def names(defaults, pos_names, pos_args, kwds): return {}.=update(defaults) \ .=update(zip(pos_names, pos_args) \ .=update(kwds)

Normally .update returns None. The reason for that is so that it's clear you are mutating an object instead of creating a new one.

And so you can't chain it, so you can only mutate one thing in a statement. People keep assuming that's an accidental unwanted side effect of the rule, but Guido explicitly saying that he doesn't like method chaining, can't you imagine it's at least possible that this is intentional, not a bug?

By using .=, it can return self, but still maintain the clarity between mutation and non-mutation.

This particular syntax is consistent with the use of OP+equal to mean mutate in place. But you might prefer, "..", or something else.

You're missing a key distinction here. OP+equal does not return self. In fact, it doesn't return _anything_, because it's not an expression at all, it's a statement. It would be very easy to make augmented assignment an expression, and even easier to make it return self (after all, it's implemented by calling dunder methods that _do_ return self!). But it wasn't designed that way. Intentionally.

The other alternative is to use a function. But it would be difficult to get the same behaviour along with the same efficiency.

How do you think a regular function would have less efficiency than an operator? Operators work by doing a complex chain of lookups to find a function to call. Ordinary calls do a simpler lookup. They both call the function the same way once they find it. You're also missing the other alternative: write it Pythonically. For example: return merge_dicts( defaults, zip(pos_names, pos_args), kwds) It's shorter, it has less extraneous syntax, it doesn't need awkward backslash continuations, and it created or modifies one value in one place. It's easier to read, and easier to reason about. Why would you want to write it the other way? Of course that merge_dicts function isn't in the stdlib. Maybe it should be. But you can write it yourself trivially. For example: def merge_dicts(*args): return {k: v for arg in args for (k, v) in dict(arg).items()} That nested comprehension might be a little too complicated; if you think so, you can split it into two expressions, or even write an explicit loop around dict.update. Whatever; this is something you write once and use every time you want to merge a bunch of dicts.

Ron Adam writes:

You would probably see it used more often like this...

def names(defaults, pos_names, pos_args, kwds): return {}.=update(defaults) \ .=update(zip(pos_names, pos_args) \ .=update(kwds)

I actually have a bunch of code in one of my apps that implements the same thing for a different reason (cascading configs), but my implementation is def names(defaults, pos_names, pos_args, kwds): for dct in pos_names, pos_args, kwds: defaults.update(dct) return defaults The other obvious use for this (as several have posted) is accumulating a sequence. In which case most uses will be well-handled with a genexp, or if you need a concrete sequence, a listcomp, and the body becomes a one (logical) liner (although it will very likely be formatted in multiple lines).

Ron Adam writes:

...

...

def names(defaults, pos_names, pos_args, kwds): return {}.=update(defaults) \ .=update(zip(pos_names, pos_args) \ .=update(kwds)

...

def names(defaults, pos_names, pos_args, kwds): for dct in pos_names, pos_args, kwds: defaults.update(dct) return defaults

Not quite the same but close. I just tried to come up with a more realistic example without having to look up a lot code. How does pos_args in your example get paired with names?

Sorry, I knew that before dinner but forgot after dinner. Same way as in yours: def names(defaults, pos_names, pos_args, kwds): for dct in zip(pos_names, pos_args), kwds: defaults.update(dct) return defaults If that doesn't work in my version (I've never used zip that way), how does it work in yours? BTW, I'd actually be more likely to write that now as def names(defaults, *updates): for update in updates: defaults.update(update) return defaults and call it with "names(zip(pos_names, pos_args), kwds)".

On 2014-02-26, at 16:28 , Ron Adam <ron3200@gmail.com> wrote:

On 02/26/2014 01:19 AM, Stephen J. Turnbull wrote:

...

Ron Adam writes:

...

...

...

def names(defaults, pos_names, pos_args, kwds): return {}.=update(defaults) \ .=update(zip(pos_names, pos_args) \ .=update(kwds)

...

...

def names(defaults, pos_names, pos_args, kwds): for dct in pos_names, pos_args, kwds: defaults.update(dct) return defaults

Not quite the same but close. I just tried to come up with a more realistic example without having to look up a lot code. How does pos_args in your example get paired with names?

Sorry, I knew that before dinner but forgot after dinner. Same way as in yours:

def names(defaults, pos_names, pos_args, kwds): for dct in zip(pos_names, pos_args), kwds: defaults.update(dct) return defaults

Yes, ok.

...

If that doesn't work in my version (I've never used zip that way), how does it work in yours? BTW, I'd actually be more likely to write that now as

def names(defaults, *updates): for update in updates: defaults.update(update) return defaults

and call it with "names(zip(pos_names, pos_args), kwds)".

The main difference between this and the one I posted is in this, defaults is mutated in your version. I'd prefer it not be.

Dictionaries are pretty flexible on how they are initiated, so it's surprising we can't do this...

D = dict(keys=names, values=args)

You can. It may not do what you want, but you definitely can do this: >>> dict(keys=names, values=args) {'keys': ['a', 'b', 'c', 'd'], 'values': [0, 1, 2]} Although you're probably looking for: >>> dict(zip(names, args)) {'a': 0, 'c': 2, 'b': 1} and if you want to do a fill because you don't have enough args: >>> dict(izip_longest(names, args, fillvalue=None)) {'a': 0, 'c': 2, 'b': 1, 'd': None} (itertools is like friendship, it's bloody magic)

The .fromkeys() method is almost that, but sets all the values to a single value. I think I would have written that a bit different.

def fromkeys(self, keys, values=None, default=None): D = {} if D is not None: D.update(zip(keys, values)] for k in keys[len(vaues):]: D[k] = default return D

And probably named it withkeys instead of fromkeys. <shrug> It's what I expected fromkeys to do.

cheers, Ron

On 02/26/2014 01:55 PM, Andrew Barnert wrote:

From: Ron Adam<ron3200@gmail.com>

Sent: Wednesday, February 26, 2014 7:28 AM

...

...

Dictionaries are pretty flexible on how they are initiated, so it's surprising we can't do this...

D = dict(keys=names, values=args)

The .fromkeys() method is almost that, but sets all the values to a single value. I think I would have written that a bit different.

def fromkeys(self, keys, values=None, default=None): D = {} if D is not None: D.update(zip(keys, values)] for k in keys[len(vaues):]: D[k] = default return D

And probably named it withkeys instead of fromkeys. <shrug> It's what I expected fromkeys to do.

Sounds like you're thinking in Smalltalk/ObjC terms, both the "with" name and the expecting two "withs":

[NSDictionary dictionaryWithObjects:values forKeys:keys]

The reason we don't need this in Python is that the default construction method takes key-value pairs, and you can get that trivially from zip:

dict(zip(keys, values))

Would it really be more readable your way?

dict.withkeys(keys, values=values)

Yes, to novices who haven't internalized zip yet. I guess the question is whether requiring people to internalize zip early is a good thing about Python, or a problem to be solved. It's not like we're requiring hundreds of weird functional idioms to make everything as brief as possible, just a very small number, each an abstraction that works consistently across a broad range of uses.

The reason I expected to be able to do that is you can get just keys, or values from a dict... dict.keys() dict.values(), and pairs.. dict.items(). It just makes sense that it will take those directly too. I'm sure I'm not the only one who thought that. I don't really buy the because we can do... ... and get... as a valid reason by it's self not to do something. Not when it's clearly related to the objects type as keys, and values are. For unrelated things, yes, it is a valid reason. For example you could say, we don't need dict.items because we can do... zip(dict.keys(), dict.values()) Or we don't need dict.keys() and dict.values because we can do... [x for x, y in dict.items()] [y for x, y in dict.itmes()] But dictionaries are used so often that having these methods really helps to make the code more readable and easy to use. (or beautiful to the eye of the programmer) In any case.. It's just my opinion. Not trying to convince anyone we need it or to do it. If it was really needed, we'd have it already. (although that argument isn't very strong either. ;-) Cheers, Ron

On Thu, Feb 27, 2014 at 8:17 AM, David Mertz <mertz@gnosis.cx> wrote:

On Wed, Feb 26, 2014 at 12:54 PM, Ron Adam <ron3200@gmail.com> wrote:

...

...

For example you could say, we don't need dict.items because we can do...

zip(dict.keys(), dict.values())

Have we actually been promised that d.keys() and d.values() walk the (unordered) dictionary in the same order, for every Python implementation/version? While I think it is almost certainly true in practice, I haven't where this invariant is guaranteed:

Yes, it's promised. It's intrinsic to the definition of keys()/values(). http://docs.python.org/3.4/library/stdtypes.html#dict-views """ Keys and values are iterated over in an arbitrary order which is non-random, varies across Python implementations, and depends on the dictionary’s history of insertions and deletions. If keys, values and items views are iterated over with no intervening modifications to the dictionary, the order of items will directly correspond. This allows the creation of (value, key) pairs using zip(): pairs = zip(d.values(), d.keys()). Another way to create the same list is pairs = [(v, k) for (k, v) in d.items()]. """ http://docs.python.org/2/library/stdtypes.html#mapping-types-dict """ If items(), keys(), values(), iteritems(), iterkeys(), and itervalues() are called with no intervening modifications to the dictionary, the lists will directly correspond. This allows the creation of (value, key) pairs using zip(): pairs =zip(d.values(), d.keys()). The same relationship holds for the iterkeys() and itervalues() methods: pairs = zip(d.itervalues(), d.iterkeys()) provides the same value for pairs. Another way to create the same list is pairs = [(v,k) for (k, v) in d.iteritems()]. """ The latter has a "CPython implementation detail" box immediately above the piece I quoted, which in a way emphasizes the fact that the bit not in the box is not CPython-specific. Note the criteria, by the way. You have to not modify the dictionary in any way in between. I believe CPython will maintain iteration order as long as the set of keys never changes, but it would be legal for a compliant Python implementation to assert-fail here: d = {1:2, 3:4, 5:6, 7:8, 9:10} items = list(d.items()) d[3] = 4 # Breakage assert items == list(d.items()) However, retrieval is safe. A compliant Python will never assert-fail if the breakage line is changed to: spam = d[3] # No breakage So a splay tree implementation (say) would have to have some other means of iterating, or it would have to not adjust itself on reads. ChrisA

Ron Adam wrote:

Dictionaries are pretty flexible on how they are initiated, so it's surprising we can't do this...

D = dict(keys=names, values=args)

All keywords are taken by the dict(name = value, ...) constructor, so this is not so surprising. But you can write that as D = dict(zip(names, values)) -- Greg

On 02/22/2014 03:42 PM, Masklinn wrote:

Executing the method always has the same effect on its subject. That it may not be used for the same purpose is a different issue and common: a[k] = v can be used to add a new (k, v) pair or to update a key to a new value (in fact Erlang's new map construct makes the difference and provides for an update-only version).

You are right! and this is a weakness of python in my view. But you could as well have chosen plain assignment, couldn't you? works the same way: n = 1 # symbol creation / definition n = 2 # symbol change / redefinition [Note, as a side-point, that if def / redef signs were different, there would be no ambiguity around local vs global scope. The ambiguity lies in fact in that the compiler cannot know if one wants to create a local symbol or change a global one. As for locally creating a global symbol, this should just not exist ;-); symbols live in their creation scope.] d

This can be done with an external wrapper, so it might be possible to do this with MacroPy. It absolutely must be a compact notation, though.

Something like: merged = c[ my_dict.update(dict_a).update(dict_b) ] Desugaring into # top-level somewhere elsedef fresh_name(x, *args): y = x for op, a, kw in args: y = getattr(y, op)(*a, **kw) y = x if y is None else y return y merged = fresh_name(my_dict, ("update", [dict_a], {}), ("update", [dict_b], {})) could be doable pretty easily, and is flexible enough to make old-school chaining work while also letting things that return None do the right thing. If you don't want to use macros, an alternative could be doing something like: chain(my_dict).update(dict_a).update(dict_b).get Using a wrapper as Steven mentioned. On Fri, Feb 21, 2014 at 5:13 PM, Steven D'Aprano <steve@pearwood.info>wrote:

On Sat, Feb 22, 2014 at 04:30:03AM +1100, Chris Angelico wrote:

...

Yeah, I'm insane, opening another theory while I'm busily championing a PEP.

Completely raving bonkers :-)

...

In Python, as in most languages, method chaining requires the method to return its own object. [...]

Rather than add syntactic support for this, I'd prefer a wrapper in the standard library. Syntax implies to me that chaining operators is, in some sense, a preferred idiom of the language, and although method chaining is sometimes useful, I don't think that it ought to be preferred. There's a difference between "Python supports this, use this syntax" and "You can do this in Python, import this library and call this function to make it happen".

[...]

...

So here's the proposal. Introduce a new operator to Python, just like the dot operator but behaving differently when it returns a bound method. We can possibly use ->, or maybe create a new operator that currently makes no sense, like .. or .> or something. Its semantics would be:

1) Look up the attribute following it on the object, exactly as per the current . operator 2) If the result is not a function, return it, exactly as per current. 3) If it is a function, though, return a wrapper which, when called, calls the inner function and then returns self.

When you say "a function", do you actually mean a function? What about other callables, such as methods (instance, class or static) or types? How does this interact with the descriptor protocol?

...

This can be done with an external wrapper, so it might be possible to do this with MacroPy. It absolutely must be a compact notation, though.

Here's an alternative: add a `chained` wrapper to, oh, let's say functools (I don't think it needs to be a built-in). In my hierarchy of language preferredness, this suggests that while Python *supports* method chaining, it isn't *preferred*. (If it were preferred, it would happen by default, or there would be syntax for it.) If you want to chain methods, you can, but it's a slightly unusual thing to do, and the fact that you have to import a module and call a wrapper class should be sufficient discouragement for casual (ab)use.

Inspired by the Ruby "tap" method, I wrote this proof-of-concept last year:

https://mail.python.org/pipermail/python-list/2013-November/660892.html

http://code.activestate.com/recipes/578770-method-chaining/

Of course, if the class author wants to support method chaining, they can always write the methods to return self :-)

-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Sat, Feb 22, 2014 at 03:31:35PM +1100, Chris Angelico wrote:

...

Here's an alternative: add a `chained` wrapper to, oh, let's say functools (I don't think it needs to be a built-in). In my hierarchy of language preferredness, this suggests that while Python *supports* method chaining, it isn't *preferred*. (If it were preferred, it would happen by default, or there would be syntax for it.) If you want to chain methods, you can, but it's a slightly unusual thing to do, and the fact that you have to import a module and call a wrapper class should be sufficient discouragement for casual (ab)use.

Downside of that is that it's really verbose. This sort of thing is useful only if it's short. It's like doing a filtered iteration:

It's not that long: chained() only adds nine characters, and if you're worried about that, call it chain() instead (seven). With your syntax, every dot lookup takes two chars instead of one, so it only takes eight method calls before my syntax is shorter than yours. # unrealistically short method names just so the example fits on one line obj->f()->g()->h()->i()->j()->k()->l()->m() chain(obj).f().g().h().i().j().k().l().m() In practice, I would expect that you would want to split the chain across multiple lines, just for readability, and if you're approaching a chain ten methods long, your code probably needs a rethink.

for spam in can if tasty: eat(spam)

for spam in filter(lambda x: tasty, can): eat(spam)

The number of characters isn't really relevent there. If that was written: for spam in f(len, can): eat(spam) which is shorter than "for spam in can if len(spam)", people would still want the filtered syntax. The problem is to think in terms of higher-order functions, and that doesn't come easily to most people.

Using filter() is so verbose that there has been and will continue to be a stream of requests for a "filtered for loop" syntax. Having to pass everything through a named wrapper is too wordy to be useful.

Not so -- if the alternative is: obj.f() obj.g() obj.h() obj.i() obj.j() obj.k() obj.l() obj.m() my version with chain() or chained() or even enable_chained_methods() wins hands down. Given that Guido dislikes chaining methods, I don't think he'll want to make it too easy to chain methods :-) -- Steven