This is more of a doubt than a new idea. Python has always worked intuitively but this was a bummer. A list has an append method. So I can do list.append(value). I tried doing list(range(10)).append(10) and it returns None. I'd usually assume list(range(10)) returns a list, to which I can append whatever I want. I tried this with List comprehension also. Doesn't work there either. Why doesn't this work? Moreover, shouldn't it work? How do I add that feature in Python?
[M Siddharth Prajosh <sprajosh@gmail.com>]
Yes. Most methods that mutate an object return None.
I'd usually assume list(range(10)) returns a list,
It does.
to which I can append whatever I want.
And you can.
I tried this with List comprehension also. Doesn't work there either. Why doesn't this work?
It does work :-) list(range(10)) created an anonymous list, then .append(10) appended 10 to that anonymous list, and list.append _always_ returns None. Since the list wasn't bound to any name, it became trash (unreachable garbage) as soon as the .append() ended, so the list was thrown away. Give it a name for clarity:
Moreover, shouldn't it work? How do I add that feature in Python?
Better instead to learn how Python works here - it's not broken :-)
I'm not excited about suggesting the walrus operator when people want to chain mutating method calls like this. It results in ugly code with way too many parentheses and a distinctly un-Pythonic flavor. I hope the OP doesn't go off and infect a whole subcommunity with this idiom. On Sat, Jan 18, 2020 at 10:05 PM Inada Naoki <songofacandy@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
The "fluent interface" (https://en.wikipedia.org/wiki/Fluent_interface) is popular in many programming languages, including in the sort of "mini-language" Pandas, within portion. But it is definitely not Pythonic. The Wikipedia article even shows how you *could* do it in Python, but mentions that Guido discourages it. What we get instead is a clear divide between mutating methods on collections that (almost) always return None, and functions like sorted() and reversed() that return copies of the underlying collection/iterable. Of course, there are many methods that don't have functions matching them. Python could have been designed differently, but using the consistency it follows is best. On Sun, Jan 19, 2020, 1:17 PM Siddharth Prajosh <sprajosh@gmail.com> wrote:
[David Mertz <mertz@gnosis.cx>]
For a bit of history that I may have made up (heh - memory fades over time!), as I recall, the very first Python pre-releases echoed to stdout every non-None statement result. So, e.g., for i in range(5): i displayed the same as the current for i in range(5): print(i) But one prolific early user loved chaining mutating method calls, each returning `self`, and so their output was littered with crap they didn't want to see. They didn't want to prefix every computational statement with, e.g., "ignore = ", so Guido stopped the magical output. Perhaps surprisingly, few people noticed the difference. But that may in large part be due to that there were few people, period. Or I'm just hallucinating again :-)
Sounds like a hallucination or fabrication. The behavior of `for i in range(10): i` in the REPL exists to this day, and list.append() never returned a value. The only thing I'm only 90% sure of is whether the REPL always ignored None values. On Sun, Jan 19, 2020 at 11:58 AM Tim Peters <tim.peters@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
[Guido]
Sounds like a hallucination or fabrication.
Nope! Turns out my memory was right :-)
The behavior of `for i in range(10): i` in the REPL exists to this day, and list.append() never returned a value.
Sure, but those weren't the claims. The claim was that the result of an expression statement was automatically printed unless it was None. `for i in range(10): i` _used_ to print 10 values _even when run from a program_ instead of from a shell. I wasn't clear about that distinction before. From Misc/HISTORY: """ ==> Release 1.0.2 (4 May 1994) <== ... * The result of a statement-level expression is no longer printed, except_ for expressions entered interactively. Consequently, the -k command line option is gone. """ Going back more: """ ==> Release 0.9.9 (29 Jul 1993) <== ... * New option -k raises an exception when an expression statement yields a value other than None. """ Now I even recall the name of the early method-chaining user whose complaints triggered those changes - but will let the past rest in peace ;-)
The only thing I'm only 90% sure of is whether the REPL always ignored None values.
I'm sure of that: it never showed None values. Because, if it had, I would have remembered bitching about the endless annoyance ;-)
I should really upgrade to 1.02! % python Python 1.0.1 (Jul 15 2016) Copyright 1991-1994 Stichting Mathematisch Centrum, Amsterdam
On Sun, Jan 19, 2020 at 4:33 PM Tim Peters <tim.peters@gmail.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
Heh. That was such a misfeature that I had thoroughly suppressed any memory of its existence. -k indeed. :-) On Sun, Jan 19, 2020 at 1:33 PM Tim Peters <tim.peters@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
[Guido, on Pythons before 1.0.2 always printing non-None expression statement results]
Heh. That was such a misfeature that I had thoroughly suppressed any memory of its existence. -k indeed. :-)
I prefer to think of it as a bit of genius :-) The natural desire to avoid mounds of useless output taught all of us, core developers and end users, to code mutating methods as procedures (always return None) rather than functions. The thing that amazed me at the time (and still does with hindsight!) is how very few complaints that attracted! Because it really was a horrible misfeature ;-)
On Sun, Jan 19, 2020 at 3:10 PM Tim Peters <tim.peters@gmail.com> wrote:
My guess is that method chaining wasn't in widespread use, and languages that made a distinction between functions and procedures (a la Pascal and Fortran) were more common than they are now. One thing that we lost when we dropped this: the bug where a user wrote `foo` rather than `foo()` was caught more reliably then. Similarly, in the async world, there's a common mistake where users write `foo()` rather than `await foo()`, and that behavior would have caught such bugs. Now we can't even flag such things statically (e.g. in mypy) because people might complain that they *meant* to write what they wrote (since it works). -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Jan 19, 2020, at 15:20, Guido van Rossum <guido@python.org> wrote:
Personally, I love the fact that in Python, the way to reference a function foo or a method baz on bar is just `foo` or `bar.baz`, and the way to call that thing later is just parens. There’s no `&foo`, much less `bar` and `&bar::baz` as separate arguments or passing the method name as a string (or inventing a new symbol or selector type that’s almost the same as a string but not) or wrapping it in a useless lambda or block, etc. as in Ruby, Smalltalk, JS, C++, etc. Sure, it means you can’t have no-parens calling syntax (unless you curry all functions a la Haskell, but then it’s very hard to have variable-signature functions) or flag accidental no-parens calls as errors. But being able to refer to all things, even functions and methods, with the one and only one obvious spelling for each thing is nice. And linters and analyzers do already flag the most common error case, where you just put `foo` instead of `foo()` on a like by itself (because it’s an expression that has no side effects but you don’t use its value). It’s only when you accidentally do something like `bar = foo` instead of `bar = foo()` (and then don’t use `bar` in a way that would detectably cause a TypeError) that can’t be detected.
On Sun, Jan 19, 2020 at 8:16 PM Andrew Barnert <abarnert@yahoo.com> wrote:
Yes of course. :-) And linters and analyzers do already flag the most common error case, where
Actually, `bar = foo` is more likely to be detected by mypy, since it is probably a type error (unless bar isn't used or is only used in an `Any` context). For mypy, flagging bare `foo` is harder, because we would have to introduce a fairly arbitrary rule that says "a top-level expression cannot have a callable type". So we haven't done this yet. (Ditto for omitting `await`, for which we would have to have a rule "a top-level expression cannot have an awaitable type".) -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Jan 19, 2020, at 20:54, Guido van Rossum <guido@python.org> wrote:
Actually, `bar = foo` is more likely to be detected by mypy, since it is probably a type error (unless bar isn't used or is only used in an `Any` context). For mypy, flagging bare `foo` is harder, because we would have to introduce a fairly arbitrary rule that says "a top-level expression cannot have a callable type". So we haven't done this yet.
Yeah, that’s a rule that makes sense for linters, but not for compilers, because it obviously can have false positives, it’s just that useful false positives are incredibly rare compared to mistakes. When I said linters and static analyzers I meant linters and linter-like static analyzers like clang-analyze, not linters and compiler/prover-like static analyzers like mypy. I can see why that was misleading, because Python has none of the former and a very prominent one of the latter; sorry for being confusing. Anyway, the point I intended to make is that it is a real cost that catching bare `foo` where `foo()` was meant can only be done by a linter, but it’s nowhere near as big as the benefit of being able to pass functions and methods around the most obvious way.
Well, mypy is often used primarily as a linter on steroids. Anyway, agreed with your point! On Tue, Jan 21, 2020 at 10:22 AM Andrew Barnert <abarnert@yahoo.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Sun, Jan 19, 2020 at 2:57 PM Tim Peters <tim.peters@gmail.com> wrote:
I generally like the result. I used some fluent interfaces long before the term was coined in 2005. In fact, several years before I used Python first in 1998. And I still sometimes really enjoy doing those Pandas-style long method chained operations. In my personal experience, I think I first used the style in some XBase-family languages that added OOP extensions (I really miss those, actually; Clipper was a really elegant language that not so many folks grew up with as various others). In Pandas it makes some sense, and is even the mostly preferred style to use. So much so there that there is talk of deprecating the `inplace=True` option that exists on most DataFrame methods. But Pandas very much eschews lops and focuses on vectorized operations. For those, I think the case for fluent interface is more compelling. In contrast, in pure Python, most of that you do is in loops over the elements of collections. In that case is does a good job of drawing your eye to the fact that a method is called but not assigned to anything. When I see `mylist.append(foo)` on a line by itself, the absence of an assigned variable drives home "this is basically a statement, called for side-effects" (yes, I know in the syntax it is an expression not a statement; but my looser cognitive model works that way). If all those mutating methods returned something meaningful, it would seem less obvious whether I was looking for the modified version of the collection or the returned thing... and whether those two would be identical to each other or not. With .append() we use it so much that it would be very familiar. But other methods of collections are "once a month" or "once a year" friends, not "ten times a day" ones... and one might forget. Yours, David... -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Jan 19, 2020, at 12:15, David Mertz <mertz@gnosis.cx> wrote:
In contrast, in pure Python, most of that you do is in loops over the elements of collections. In that case is does a good job of drawing your eye to the fact that a method is called but not assigned to anything. When I see `mylist.append(foo)` on a line by itself, the absence of an assigned variable drives home "this is basically a statement, called for side-effects" (yes, I know in the syntax it is an expression not a statement; but my looser cognitive model works that way).
But it is an expression statement. And that works just as well without having to be mildly incorrect, and it extends even further from there. Most statements in most Python code mutate one thing. Most of them explicitly mutate the first thing mentioned (and any expressions within them are usually nonmutating). Expression statements dont explicitly mutate anything, but usually at the top level they’re usually either a method call that mutates its self, or a function call that mutates something implicit in the function, and subexpressions are usually nonmutating. So, most statements in most Python code mutate the first thing: # the only mutation is (probably) to spam: spam = eggs(cheese) spam += eggs spam[eggs] = cheese spam.eggs(cheese) # the only mutation is (probably) to the stdout implicit in print: print(spam) print(spam.eggs(cheese)) print(spam(eggs)) There are of course exceptions. A from … import statement creates the names at the end of the statement rather than the start. Mutating methods are only encouraged to return None, not required. And so on. But the rough guideline works well enough for skimming most Python code to see “where does x get changed” in a way that doesn’t work for languages that encourage fluent mutating style, like JS or Ruby or C++. What about the walrus operator? The fact that it clearly breaks this is, I think, a big part of the reason that so many people were against it. But the fact that it’s usually used only to mutate something that’s only relevant within the statement is a big part of the reason people actually like it once they see real examples.
On Sun, Jan 19, 2020 at 11:59 AM Tim Peters <tim.peters@gmail.com> wrote:
which is what MATLAB does, if you don't put a semi-colon at the end of the line (or somethign like that it's been a LONG time) I always found it mostly annoying, though it did make "print debugging" really easy. - CHB Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
By the way, if anyone is actually interested in this other than for nostalgia , there have been a number of lengthy discussion on this list about "Fluent" interfaces for Python. LIke this one for instance: https://groups.google.com/forum/#!searchin/python-ideas/Method$20Chaining%7C... -CHB On Sun, Jan 19, 2020 at 5:29 PM Christopher Barker <pythonchb@gmail.com> wrote:
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Sun, Jan 19, 2020 at 02:37:14PM -0500, David Mertz wrote:
I can't speak about Pandas, but what the OP is expecting to do is not a fluent interface. It's just method chaining. We do method chaining all the time with immutable "value objects" like strings: result = mystring.upper().strip().center(width).encode('utf-8') and nobody blinks an eye or calls it unpythonic. A fluent interface is built on top of method chaining: you can't (easily?) have a fluent interface without chaining, but you can have chaining without a fluent interface. See Fowler, who came up with the term and ought to know what he meant by it :-) https://www.martinfowler.com/bliki/FluentInterface.html In other words: merely returning "self" from your mutating methods does not make a fluent interface. -- Steven
On Sun, Jan 19, 2020 at 3:56 PM Steven D'Aprano <steve@pearwood.info> wrote:
I'm not sure what you intend here: A fluent interface is normally implemented by using method chaining <https://en.wikipedia.org/wiki/Method_chaining> to implement method cascading <https://en.wikipedia.org/wiki/Method_cascading> (in languages that do not natively support cascading), concretely by having each method return this <https://en.wikipedia.org/wiki/This_(computer_programming)> ( self). While there are other things that might make a fluent interface, method chaining is by far the most commonly used approach.
result = mystring.upper().strip().center(width).encode('utf-8')
But yes, there are places in Python where "fluency" is the normal style. I was careful to say "collections" and "mutate" in my prior comments. A string is a kinda-sorta collection, but it's definitely not mutable.
I don't really see what you mean in Fowler that would allow "non-fluent" method chaining, but I'll stipulate there's some sentence I missed or read wrong. Nowadays, the term is in more widespread use, and exactly what Fowler meant in 2005 isn't really essential. For example, I definitely think of pipes between Unix commands using the shell as a "fluent interface" (that I like very much). That's not method calls, but it is absolutely *chaining*. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sun, Jan 19, 2020 at 04:27:30PM -0500, David Mertz wrote:
I'm not sure what you intend here:
A fluent interface is normally implemented by using method chaining
Indeed. *Implemented by*, not "the same as".
If you read Fowler's post on the subject all the way to the end, you will see his comment: I've also noticed a common misconception - many people seem to equate fluent interfaces with Method Chaining. Certainly chaining is a common technique to use with fluent interfaces, but true fluency is much more than that. He also points out that returning self (this) is not a hard requirement: You should choose your return type based on what you need to continue fluent action. Here's the link again for anyone who missed it the first time. https://www.martinfowler.com/bliki/FluentInterface.html Even the Wikipedia page you linked to says: Note that a "fluent interface" means more than just method cascading via chaining; it entails designing an interface that reads like a DSL, using other techniques like "nested functions and object scoping". which is one paragraph after the bit you quoted at me. The bottom line is that what the OP was asking for: result = list(something).append(spam).append(eggs).append(cheese) is not a fluent interface, it's just boring old method chaining like we already do with strings. Fowler's example of a fluent interface is worth reading: he uses it to build a compound object (an Order) using a lightweight DSL. The critical part is Fowler's comment: Probably the most important thing to notice about this style is that the intent is to do something along the lines of an internal DomainSpecificLanguage. Indeed this is why we chose the term 'fluent' to describe it, in many ways the two terms are synonyms. The API is primarily designed to be readable and to flow. If it doesn't read fluently as a domain specific language, it isn't a fluent interface. Merely chaining function calls or method calls or infix operators or statements is not sufficient. If it were sufficient, then everything is a fluent interface and the term is meaningless.
Are you arguing that *any* time you chain two method call in a single expression, that's a fluent interface? If not, then you can answer that question yourself: think of a case where chained methods aren't sufficient, and you have your answer: string.strip().split().count("spam") That's just a mechanical rearrangement of postfix or prefix functional syntax with no additional DSL features. string strip split "spam" count count(split(strip(string)))("spam") It's not a DSL and it's not "fluent" in any meaningful sense, only in the trivial sense that we can read source code in languages we can read. Mutability is also a side-issue. Being written in Java, Fowler's "Order" example would probably mutate a Customer object and an Order object, but one could use a fluent interface with Haskell where each call created a new object and disgarded the previous one. The aim of fluent interfaces is to create a domain-specific interface, not just to squeeze things into a single expression by chaining dots. The *mechanism* they use to implement that DSL is method chaining together with objects designed to cooperate together to give the semantics needed in your domain. For example, in Fowler's example, the "with" method knows about products, and can create an order line and add it to an order. In the same way that people mistakenly think that the Rule of Demeter is about "never put two dots in the one expression", people are mistaken if they think fluent interfaces are *merely* about chaining methods. It's like arguing that Python's command line and REPL is a GUI, because the terminal window is displayed using pixels. Well yes it is, but that's not sufficient to make a GUI.
The DSL part gets to the core of what distinguishes a fluent interface from merely chaining methods (or pipes, or function calls). If there's no difference between "fluent interface" and "method chaining" then why do we need two terms for them, and how can we say that one is implemented with the other? -- Steven
It’s not even immutable that’s the key thing here, but nonmutating. Even on mutable types, Python encourages you to chain up nonmutating operations. And it goes beyond linear chaining; you can compose any nonmutating method calls, function calls, operators, and comprehensions you want. And it only becomes unpythonic it if gets too complicated to eye-parse. But you can’t compose mutating method calls, you have to imperatively chain them up as separate statements. You can even see this in the OP’s problem. He didn’t want to mutate list(range(10)); that’s a garbage value. But he did want to get the value that’s list(range(10)) with the list [10] stuck on the end, and Python does make that trivial: list(range(10)) + [10] You might not want to do that if 10 is an arbitrarily large user-supplied value, because that extra copy could be a serious performance cost. But in most such cases the list is a serious performance cost in the first place, and you really just wanted to use the range itself as an Iterable and itertools.chain it. When you have a nonmutating algorithm, you usually can compose it up roughly the same way you would in Haskell; when you have an in-place algorithm you usually can write it as a chain of imperative statements roughly the same way you would in C. Python only gets in your way when you have a mishmash of the two where it would make sense to write confusing things like `lst.sort() + lst.remove(10)` or a comprehension that mutates as it goes. Good uses of that kind of mishmash do exist, but aren’t that common.
On Sun, Jan 19, 2020 at 08:32:43AM -0800, Guido van Rossum wrote:
It is worse than that: it doesn't even solve the OP's problem of wanting to chain method calls. py> (xs := list(range(10))).append(42) # seems to work py> xs [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 42] py> (xs := list(range(10))).append(42).append(999) Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'NoneType' object has no attribute 'append' -- Steven
[M Siddharth Prajosh <sprajosh@gmail.com>]
Yes. Most methods that mutate an object return None.
I'd usually assume list(range(10)) returns a list,
It does.
to which I can append whatever I want.
And you can.
I tried this with List comprehension also. Doesn't work there either. Why doesn't this work?
It does work :-) list(range(10)) created an anonymous list, then .append(10) appended 10 to that anonymous list, and list.append _always_ returns None. Since the list wasn't bound to any name, it became trash (unreachable garbage) as soon as the .append() ended, so the list was thrown away. Give it a name for clarity:
Moreover, shouldn't it work? How do I add that feature in Python?
Better instead to learn how Python works here - it's not broken :-)
I'm not excited about suggesting the walrus operator when people want to chain mutating method calls like this. It results in ugly code with way too many parentheses and a distinctly un-Pythonic flavor. I hope the OP doesn't go off and infect a whole subcommunity with this idiom. On Sat, Jan 18, 2020 at 10:05 PM Inada Naoki <songofacandy@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
The "fluent interface" (https://en.wikipedia.org/wiki/Fluent_interface) is popular in many programming languages, including in the sort of "mini-language" Pandas, within portion. But it is definitely not Pythonic. The Wikipedia article even shows how you *could* do it in Python, but mentions that Guido discourages it. What we get instead is a clear divide between mutating methods on collections that (almost) always return None, and functions like sorted() and reversed() that return copies of the underlying collection/iterable. Of course, there are many methods that don't have functions matching them. Python could have been designed differently, but using the consistency it follows is best. On Sun, Jan 19, 2020, 1:17 PM Siddharth Prajosh <sprajosh@gmail.com> wrote:
[David Mertz <mertz@gnosis.cx>]
For a bit of history that I may have made up (heh - memory fades over time!), as I recall, the very first Python pre-releases echoed to stdout every non-None statement result. So, e.g., for i in range(5): i displayed the same as the current for i in range(5): print(i) But one prolific early user loved chaining mutating method calls, each returning `self`, and so their output was littered with crap they didn't want to see. They didn't want to prefix every computational statement with, e.g., "ignore = ", so Guido stopped the magical output. Perhaps surprisingly, few people noticed the difference. But that may in large part be due to that there were few people, period. Or I'm just hallucinating again :-)
Sounds like a hallucination or fabrication. The behavior of `for i in range(10): i` in the REPL exists to this day, and list.append() never returned a value. The only thing I'm only 90% sure of is whether the REPL always ignored None values. On Sun, Jan 19, 2020 at 11:58 AM Tim Peters <tim.peters@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
[Guido]
Sounds like a hallucination or fabrication.
Nope! Turns out my memory was right :-)
The behavior of `for i in range(10): i` in the REPL exists to this day, and list.append() never returned a value.
Sure, but those weren't the claims. The claim was that the result of an expression statement was automatically printed unless it was None. `for i in range(10): i` _used_ to print 10 values _even when run from a program_ instead of from a shell. I wasn't clear about that distinction before. From Misc/HISTORY: """ ==> Release 1.0.2 (4 May 1994) <== ... * The result of a statement-level expression is no longer printed, except_ for expressions entered interactively. Consequently, the -k command line option is gone. """ Going back more: """ ==> Release 0.9.9 (29 Jul 1993) <== ... * New option -k raises an exception when an expression statement yields a value other than None. """ Now I even recall the name of the early method-chaining user whose complaints triggered those changes - but will let the past rest in peace ;-)
The only thing I'm only 90% sure of is whether the REPL always ignored None values.
I'm sure of that: it never showed None values. Because, if it had, I would have remembered bitching about the endless annoyance ;-)
I should really upgrade to 1.02! % python Python 1.0.1 (Jul 15 2016) Copyright 1991-1994 Stichting Mathematisch Centrum, Amsterdam
On Sun, Jan 19, 2020 at 4:33 PM Tim Peters <tim.peters@gmail.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
Heh. That was such a misfeature that I had thoroughly suppressed any memory of its existence. -k indeed. :-) On Sun, Jan 19, 2020 at 1:33 PM Tim Peters <tim.peters@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
[Guido, on Pythons before 1.0.2 always printing non-None expression statement results]
Heh. That was such a misfeature that I had thoroughly suppressed any memory of its existence. -k indeed. :-)
I prefer to think of it as a bit of genius :-) The natural desire to avoid mounds of useless output taught all of us, core developers and end users, to code mutating methods as procedures (always return None) rather than functions. The thing that amazed me at the time (and still does with hindsight!) is how very few complaints that attracted! Because it really was a horrible misfeature ;-)
On Sun, Jan 19, 2020 at 3:10 PM Tim Peters <tim.peters@gmail.com> wrote:
My guess is that method chaining wasn't in widespread use, and languages that made a distinction between functions and procedures (a la Pascal and Fortran) were more common than they are now. One thing that we lost when we dropped this: the bug where a user wrote `foo` rather than `foo()` was caught more reliably then. Similarly, in the async world, there's a common mistake where users write `foo()` rather than `await foo()`, and that behavior would have caught such bugs. Now we can't even flag such things statically (e.g. in mypy) because people might complain that they *meant* to write what they wrote (since it works). -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Jan 19, 2020, at 15:20, Guido van Rossum <guido@python.org> wrote:
Personally, I love the fact that in Python, the way to reference a function foo or a method baz on bar is just `foo` or `bar.baz`, and the way to call that thing later is just parens. There’s no `&foo`, much less `bar` and `&bar::baz` as separate arguments or passing the method name as a string (or inventing a new symbol or selector type that’s almost the same as a string but not) or wrapping it in a useless lambda or block, etc. as in Ruby, Smalltalk, JS, C++, etc. Sure, it means you can’t have no-parens calling syntax (unless you curry all functions a la Haskell, but then it’s very hard to have variable-signature functions) or flag accidental no-parens calls as errors. But being able to refer to all things, even functions and methods, with the one and only one obvious spelling for each thing is nice. And linters and analyzers do already flag the most common error case, where you just put `foo` instead of `foo()` on a like by itself (because it’s an expression that has no side effects but you don’t use its value). It’s only when you accidentally do something like `bar = foo` instead of `bar = foo()` (and then don’t use `bar` in a way that would detectably cause a TypeError) that can’t be detected.
On Sun, Jan 19, 2020 at 8:16 PM Andrew Barnert <abarnert@yahoo.com> wrote:
Yes of course. :-) And linters and analyzers do already flag the most common error case, where
Actually, `bar = foo` is more likely to be detected by mypy, since it is probably a type error (unless bar isn't used or is only used in an `Any` context). For mypy, flagging bare `foo` is harder, because we would have to introduce a fairly arbitrary rule that says "a top-level expression cannot have a callable type". So we haven't done this yet. (Ditto for omitting `await`, for which we would have to have a rule "a top-level expression cannot have an awaitable type".) -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Jan 19, 2020, at 20:54, Guido van Rossum <guido@python.org> wrote:
Actually, `bar = foo` is more likely to be detected by mypy, since it is probably a type error (unless bar isn't used or is only used in an `Any` context). For mypy, flagging bare `foo` is harder, because we would have to introduce a fairly arbitrary rule that says "a top-level expression cannot have a callable type". So we haven't done this yet.
Yeah, that’s a rule that makes sense for linters, but not for compilers, because it obviously can have false positives, it’s just that useful false positives are incredibly rare compared to mistakes. When I said linters and static analyzers I meant linters and linter-like static analyzers like clang-analyze, not linters and compiler/prover-like static analyzers like mypy. I can see why that was misleading, because Python has none of the former and a very prominent one of the latter; sorry for being confusing. Anyway, the point I intended to make is that it is a real cost that catching bare `foo` where `foo()` was meant can only be done by a linter, but it’s nowhere near as big as the benefit of being able to pass functions and methods around the most obvious way.
Well, mypy is often used primarily as a linter on steroids. Anyway, agreed with your point! On Tue, Jan 21, 2020 at 10:22 AM Andrew Barnert <abarnert@yahoo.com> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Sun, Jan 19, 2020 at 2:57 PM Tim Peters <tim.peters@gmail.com> wrote:
I generally like the result. I used some fluent interfaces long before the term was coined in 2005. In fact, several years before I used Python first in 1998. And I still sometimes really enjoy doing those Pandas-style long method chained operations. In my personal experience, I think I first used the style in some XBase-family languages that added OOP extensions (I really miss those, actually; Clipper was a really elegant language that not so many folks grew up with as various others). In Pandas it makes some sense, and is even the mostly preferred style to use. So much so there that there is talk of deprecating the `inplace=True` option that exists on most DataFrame methods. But Pandas very much eschews lops and focuses on vectorized operations. For those, I think the case for fluent interface is more compelling. In contrast, in pure Python, most of that you do is in loops over the elements of collections. In that case is does a good job of drawing your eye to the fact that a method is called but not assigned to anything. When I see `mylist.append(foo)` on a line by itself, the absence of an assigned variable drives home "this is basically a statement, called for side-effects" (yes, I know in the syntax it is an expression not a statement; but my looser cognitive model works that way). If all those mutating methods returned something meaningful, it would seem less obvious whether I was looking for the modified version of the collection or the returned thing... and whether those two would be identical to each other or not. With .append() we use it so much that it would be very familiar. But other methods of collections are "once a month" or "once a year" friends, not "ten times a day" ones... and one might forget. Yours, David... -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Jan 19, 2020, at 12:15, David Mertz <mertz@gnosis.cx> wrote:
In contrast, in pure Python, most of that you do is in loops over the elements of collections. In that case is does a good job of drawing your eye to the fact that a method is called but not assigned to anything. When I see `mylist.append(foo)` on a line by itself, the absence of an assigned variable drives home "this is basically a statement, called for side-effects" (yes, I know in the syntax it is an expression not a statement; but my looser cognitive model works that way).
But it is an expression statement. And that works just as well without having to be mildly incorrect, and it extends even further from there. Most statements in most Python code mutate one thing. Most of them explicitly mutate the first thing mentioned (and any expressions within them are usually nonmutating). Expression statements dont explicitly mutate anything, but usually at the top level they’re usually either a method call that mutates its self, or a function call that mutates something implicit in the function, and subexpressions are usually nonmutating. So, most statements in most Python code mutate the first thing: # the only mutation is (probably) to spam: spam = eggs(cheese) spam += eggs spam[eggs] = cheese spam.eggs(cheese) # the only mutation is (probably) to the stdout implicit in print: print(spam) print(spam.eggs(cheese)) print(spam(eggs)) There are of course exceptions. A from … import statement creates the names at the end of the statement rather than the start. Mutating methods are only encouraged to return None, not required. And so on. But the rough guideline works well enough for skimming most Python code to see “where does x get changed” in a way that doesn’t work for languages that encourage fluent mutating style, like JS or Ruby or C++. What about the walrus operator? The fact that it clearly breaks this is, I think, a big part of the reason that so many people were against it. But the fact that it’s usually used only to mutate something that’s only relevant within the statement is a big part of the reason people actually like it once they see real examples.
On Sun, Jan 19, 2020 at 11:59 AM Tim Peters <tim.peters@gmail.com> wrote:
which is what MATLAB does, if you don't put a semi-colon at the end of the line (or somethign like that it's been a LONG time) I always found it mostly annoying, though it did make "print debugging" really easy. - CHB Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
By the way, if anyone is actually interested in this other than for nostalgia , there have been a number of lengthy discussion on this list about "Fluent" interfaces for Python. LIke this one for instance: https://groups.google.com/forum/#!searchin/python-ideas/Method$20Chaining%7C... -CHB On Sun, Jan 19, 2020 at 5:29 PM Christopher Barker <pythonchb@gmail.com> wrote:
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Sun, Jan 19, 2020 at 02:37:14PM -0500, David Mertz wrote:
I can't speak about Pandas, but what the OP is expecting to do is not a fluent interface. It's just method chaining. We do method chaining all the time with immutable "value objects" like strings: result = mystring.upper().strip().center(width).encode('utf-8') and nobody blinks an eye or calls it unpythonic. A fluent interface is built on top of method chaining: you can't (easily?) have a fluent interface without chaining, but you can have chaining without a fluent interface. See Fowler, who came up with the term and ought to know what he meant by it :-) https://www.martinfowler.com/bliki/FluentInterface.html In other words: merely returning "self" from your mutating methods does not make a fluent interface. -- Steven
On Sun, Jan 19, 2020 at 3:56 PM Steven D'Aprano <steve@pearwood.info> wrote:
I'm not sure what you intend here: A fluent interface is normally implemented by using method chaining <https://en.wikipedia.org/wiki/Method_chaining> to implement method cascading <https://en.wikipedia.org/wiki/Method_cascading> (in languages that do not natively support cascading), concretely by having each method return this <https://en.wikipedia.org/wiki/This_(computer_programming)> ( self). While there are other things that might make a fluent interface, method chaining is by far the most commonly used approach.
result = mystring.upper().strip().center(width).encode('utf-8')
But yes, there are places in Python where "fluency" is the normal style. I was careful to say "collections" and "mutate" in my prior comments. A string is a kinda-sorta collection, but it's definitely not mutable.
I don't really see what you mean in Fowler that would allow "non-fluent" method chaining, but I'll stipulate there's some sentence I missed or read wrong. Nowadays, the term is in more widespread use, and exactly what Fowler meant in 2005 isn't really essential. For example, I definitely think of pipes between Unix commands using the shell as a "fluent interface" (that I like very much). That's not method calls, but it is absolutely *chaining*. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sun, Jan 19, 2020 at 04:27:30PM -0500, David Mertz wrote:
I'm not sure what you intend here:
A fluent interface is normally implemented by using method chaining
Indeed. *Implemented by*, not "the same as".
If you read Fowler's post on the subject all the way to the end, you will see his comment: I've also noticed a common misconception - many people seem to equate fluent interfaces with Method Chaining. Certainly chaining is a common technique to use with fluent interfaces, but true fluency is much more than that. He also points out that returning self (this) is not a hard requirement: You should choose your return type based on what you need to continue fluent action. Here's the link again for anyone who missed it the first time. https://www.martinfowler.com/bliki/FluentInterface.html Even the Wikipedia page you linked to says: Note that a "fluent interface" means more than just method cascading via chaining; it entails designing an interface that reads like a DSL, using other techniques like "nested functions and object scoping". which is one paragraph after the bit you quoted at me. The bottom line is that what the OP was asking for: result = list(something).append(spam).append(eggs).append(cheese) is not a fluent interface, it's just boring old method chaining like we already do with strings. Fowler's example of a fluent interface is worth reading: he uses it to build a compound object (an Order) using a lightweight DSL. The critical part is Fowler's comment: Probably the most important thing to notice about this style is that the intent is to do something along the lines of an internal DomainSpecificLanguage. Indeed this is why we chose the term 'fluent' to describe it, in many ways the two terms are synonyms. The API is primarily designed to be readable and to flow. If it doesn't read fluently as a domain specific language, it isn't a fluent interface. Merely chaining function calls or method calls or infix operators or statements is not sufficient. If it were sufficient, then everything is a fluent interface and the term is meaningless.
Are you arguing that *any* time you chain two method call in a single expression, that's a fluent interface? If not, then you can answer that question yourself: think of a case where chained methods aren't sufficient, and you have your answer: string.strip().split().count("spam") That's just a mechanical rearrangement of postfix or prefix functional syntax with no additional DSL features. string strip split "spam" count count(split(strip(string)))("spam") It's not a DSL and it's not "fluent" in any meaningful sense, only in the trivial sense that we can read source code in languages we can read. Mutability is also a side-issue. Being written in Java, Fowler's "Order" example would probably mutate a Customer object and an Order object, but one could use a fluent interface with Haskell where each call created a new object and disgarded the previous one. The aim of fluent interfaces is to create a domain-specific interface, not just to squeeze things into a single expression by chaining dots. The *mechanism* they use to implement that DSL is method chaining together with objects designed to cooperate together to give the semantics needed in your domain. For example, in Fowler's example, the "with" method knows about products, and can create an order line and add it to an order. In the same way that people mistakenly think that the Rule of Demeter is about "never put two dots in the one expression", people are mistaken if they think fluent interfaces are *merely* about chaining methods. It's like arguing that Python's command line and REPL is a GUI, because the terminal window is displayed using pixels. Well yes it is, but that's not sufficient to make a GUI.
The DSL part gets to the core of what distinguishes a fluent interface from merely chaining methods (or pipes, or function calls). If there's no difference between "fluent interface" and "method chaining" then why do we need two terms for them, and how can we say that one is implemented with the other? -- Steven
It’s not even immutable that’s the key thing here, but nonmutating. Even on mutable types, Python encourages you to chain up nonmutating operations. And it goes beyond linear chaining; you can compose any nonmutating method calls, function calls, operators, and comprehensions you want. And it only becomes unpythonic it if gets too complicated to eye-parse. But you can’t compose mutating method calls, you have to imperatively chain them up as separate statements. You can even see this in the OP’s problem. He didn’t want to mutate list(range(10)); that’s a garbage value. But he did want to get the value that’s list(range(10)) with the list [10] stuck on the end, and Python does make that trivial: list(range(10)) + [10] You might not want to do that if 10 is an arbitrarily large user-supplied value, because that extra copy could be a serious performance cost. But in most such cases the list is a serious performance cost in the first place, and you really just wanted to use the range itself as an Iterable and itertools.chain it. When you have a nonmutating algorithm, you usually can compose it up roughly the same way you would in Haskell; when you have an in-place algorithm you usually can write it as a chain of imperative statements roughly the same way you would in C. Python only gets in your way when you have a mishmash of the two where it would make sense to write confusing things like `lst.sort() + lst.remove(10)` or a comprehension that mutates as it goes. Good uses of that kind of mishmash do exist, but aren’t that common.
On Sun, Jan 19, 2020 at 08:32:43AM -0800, Guido van Rossum wrote:
It is worse than that: it doesn't even solve the OP's problem of wanting to chain method calls. py> (xs := list(range(10))).append(42) # seems to work py> xs [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 42] py> (xs := list(range(10))).append(42).append(999) Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'NoneType' object has no attribute 'append' -- Steven
participants (8)
-
Andrew Barnert -
Christopher Barker -
David Mertz -
Guido van Rossum -
Inada Naoki -
Siddharth Prajosh -
Steven D'Aprano -
Tim Peters