Ruby-style Blocks in Python Idea
Hey all, I've come up with a way to do Ruby-style blocks in what I feel to be a Pythonic way: using employees.select do (employee): if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) I originally overloaded the ``with`` keyword, but on Guido's guidance and responses from many others, switched to the ``using`` keyword. I explain the idea in detail in this blog article: http://tav.espians.com/ruby-style-blocks-in-python.html It covers everything from why these are useful to a proposal of how the new ``do`` statement and __do__ function could work. Please note that the intention of this proposal is not to encourage iteration with blocks -- Python already does this rather elegantly -- but blocks are very useful for more than just iteration as various Ruby applications have shown. Let me know what you think. My apologies if I've missed something obvious. Thanks! -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
On 3/9/2009 2:04 PM, tav wrote:
I believe this is just an extension to the lambda keyword. If lambdas could define a block, not just a statement, this would e.g. be employees.select( lambda employee: if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) ) or tmp = lambda employee: if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) employees.select(tmp) I see no reason for introducing two new keywords to do this, as you are really just enhancing the current lambda keyword. On the other hand, turning blocks into anonymous functions would be very useful for functional programming. As such, I like your suggestion. This also has a great potential for abuse (as in writing unreadable code), just consider how anonymous classes are used in Java's GUI toolkits. I really don't want to see self.Bind(wx.BUTTON, lamda: evt <some huge block of code here> , mybutton) in wxPython code. But Java programmers coming to Pytho would jump to this, as they have been brain washed to use anonymous classes for everything (no pun intended). Sturla Molden
Hey Stuart,
I believe this is just an extension to the lambda keyword. If lambdas could define a block, not just a statement, this would e.g. be
Multi-line lambdas would be nice, but I struggle to find a way to do so in a Pythonic manner. See: http://unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html It would be nice to find a way though...
You might as well just use ``def`` above...
I see no reason for introducing two new keywords to do this, as you are really just enhancing the current lambda keyword.
I agree that two new keywords is a bit much. I tried to re-use ``with`` initially -- but I guess people would be confused by the conflicting semantics.
On the other hand, turning blocks into anonymous functions would be very useful for functional programming. As such, I like your suggestion.
Thanks =) -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
On Mon, Mar 09, 2009, Sturla Molden wrote:
There's a substantial minority (possibly even a majority) in the Python community that abhors functional programming. Even among those who like functional programming, there's a substantial population that dislikes extensive use of anonymous functions. The trick to getting features for functional programming accepted is to make them look as Pythonic as possible. Right now, I'm somewhere between -0 and -1 on this proposal, because all the motivation I see looks like it's perfectly satisfied by using ``def`` instead of lambda. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson
Hey Aahz,
The trick to getting features for functional programming accepted is to make them look as Pythonic as possible.
I spent considerable effort to make the using/do statement as Pythonic as possible. Could you please elaborate on what you don't like about it? Please note that the lambda thing was Sturla's follow up comment... -- Thanks, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
On 3/9/2009 4:53 PM, tav wrote:
If I can elaborate as well. There are three things I don't like: 1. You are introducing two new keywords. Solving problems by constantly adding new syntax is how programming languages are designed in Redmond, WA. I don't exactly know what Pythonic means, but bloating the syntax is not. 2. Most of this is covered by 'def'. Python allows functions to be nested. Python does support closures. 3. Anonymous classes in Java have more cases for abuse than use. Just see how they are abused to write callbacks/handlers. They are a notorious source of unreadable and unmaintainable spaghetti code. S.M.
On Mon, Mar 09, 2009, tav wrote:
That was a general point. The specific point was what you cut: your proposal seems to offer little advantage over a ``def``. You need to justify yourself more thoroughly. Also, because this list is archived, you should probably include your entire argument here rather than referring to an external web page. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson
Hallöchen! tav writes:
Two things for me: The "using ..." is not well-readable. The "(employee)" sits clumsily at the end of the line whithout any connection to the rest. Even worse, the "do" anticipates the ":", which in Python already means "do". And secondly, I'm not comfortable with the fact that the return value is the first (or last? or all?) expression the interpreter stumbles over. Because Python distinguishs between expressions and statements, you have to look twice to see what actually happens in the block. In other words, expressions work differently in these almost-functions, so we end up with two kinds of functions that have different semantic rules. This makes reading more difficult, as well as code-reuse. Tschö, Torsten. -- Torsten Bronger, aquisgrana, europa vetus Jabber ID: torsten.bronger@jabber.rwth-aachen.de
On 3/9/2009 4:45 PM, Aahz wrote:
There's a substantial minority (possibly even a majority) in the Python community that abhors functional programming.
There are a substantial minority that use Python for scientific computing (cf. numpy and scipy, the Hubble space telescope, the NEURON simulator, Sage, etc.) For numerical computing, functional programming often leads to code that are shorter and easier to read. That is, equations look like functions, not like classes. S.M.
On Mon, Mar 09, 2009, Sturla Molden wrote:
Yes, I know; I'm just pointing out that Python is not a pure functional language and that there's a tension within the community about how far Python should go in the functional direction. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson
On Tue, 10 Mar 2009 03:24:36 am Sturla Molden wrote:
I don't understand what you mean. As far as I can see, equations never look like classes in Python, regardless of whether you are using functional programming, object-oriented programming or procedural programming. Can you give me an example of what you mean? Secondly, the proposal relates to *anonymous* functions, which is a small part of functional programming. Perhaps they are necessary in a purely functional programming language, but Python is not such a language. Python never *requires* anonymous functions, they are a convenience and that's all. -- Steven D'Aprano
Aahz wrote:
I am not one of them, if there really are such.
Like many other Pythonistas I recognize that that an uninformative stock name of '<lambda>' is defective relative to an informative name that points back to readable code. What I dislike is the anonymity-cult claim that the defect is a virtue. Since I routinely use standard names 'f' and 'g' (from math) to name functions whose name I do not care about, I am baffled (and annoyed) by (repeated) claims such as "Having to name a one-off function adds additional cognitive overload to a developer." (Tav). Golly gee, if one cannot decide on standard one-char name, how can he manage the rest of Python? (I also, like others, routinely use 'C' for class and 'c' for C instance. What next? A demand for anonymous classes? Whoops, I just learned that Java has those.) But I have no problem with the use of lambda expressions as a convenience, where appropriate. Terry Jan Reedy
On Mon, Mar 9, 2009 at 4:09 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Andrew Koening once gave me a good use case where lambdas are really a lot more convenient than named functions. He was initializing a large data structure that was used by an interpreter for some language. It was a single expression (probably a list of tuples or a dict). Each record contained various bits of information (e.g. the operator symbol and its precedence and associativity) as well as a function (almost always a very simple lambda) that implemented it. Since this table was 100s of records long, it would have been pretty inconvenient to first have to define 100s of small one-line functions and give them names, only to reference them once in the initializer. This use case doesn't have a nice equivalent without anonymous functions (though I'm sure that if there really was no other way it could be done, e.g. using registration=style decorators). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
With a small trick you don't need lambdas for this, if the keys are python identifiers. If they aren't you can add the real key to the info tuple and then generate a new dict in a oneliner. But yeah, it's a bit ugly/abusive. But it is possible without defining parts of the tuple at different places! :) def info(*args): def wrapper(f): return args + (f,) return wrapper def mkdict(): @info("a",12,9.9,None) def foo(): print "this is foo" @info("b",23,0.0,foo) def bar(): print "this is bar" @info("c",42,3.1415,None) def baz(): print "this is baz" return locals() print mkdict() -panzi
Guido van Rossum wrote:
Initializing such structures is one of the use cases I intended under 'where appropriate'. Adding more powerful expressions, like comprehensions (and g.e's) that do not break Python's basic syntactic model of mixed expressions and indented statements has added to the convenience.
The convenience is from having function expressions. If the expression syntax allowed the optional attachment of a name, it would be just as convenient. In some cases, I am sure people would find it even more convenient if they could add in a name, especially when there is nothing else in the structure to serve as a substitute. 'Anonymous' is a different concept from 'expression-defined' despite the tendency to conflate the two. Terry Jan Reedy
On Tue, Mar 10, 2009 at 1:15 PM, Terry Reedy <tjreedy@udel.edu> wrote:
If I read you correctly you're saying that having an expression that returns a function (other than referencing it by name) is not the same as having anonymous functions. This sounds like quite the hairsplitting argument. Why is it important to you to split this particular hair? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 3/10/09, Guido van Rossum <guido@python.org> wrote:
On Tue, Mar 10, 2009 at 1:15 PM, Terry Reedy <tjreedy@udel.edu> wrote:
On Mon, Mar 9, 2009 at 4:09 PM, Terry Reedy <tjreedy@udel.edu> wrote:
... What I dislike is the anonymity-cult claim that the defect is a virtue.
'Anonymous' is a different concept from 'expression-defined' despite the tendency to conflate the two.
An expression that *creates* and returns a function is useful. A way to create unnamed functions may or may not be useful. Right now, the two are tied together, as lambda is the best way to do either. Mentally untangling them might lead to better code. If the name in a def were optional, it would meet the perceived need for anonymity, but still wouldn't meet the need for creating and returning a function within a single expression. # Would this really ever be useful? # Not to me, but the anon-lovers suggest yes. # Cognition difference, or just confounding the two uses of lambda? def (a): return a+3 On the other hand, if def became an expression, it would meet the need for function-creating expressions (and would have at least reduced the need for decorators). add_callback(button1, def add3(a): return a+3) (And yes, I understand that there are reasons why class, def, and import do not return values, even if I sometimes wish they did.) -jJ
On Tue, Mar 10, 2009 at 3:22 PM, Jim Jewett <jimjjewett@gmail.com> wrote:
I'm feeling really dense right now -- I still don't see the difference between the two. Are you saying that you would prefer an expression that creates a *named* function? That seems to be really bizarre -- like claiming that you don't like expressions that return anonymous numbers.
Moreover, unless you used a decorator, there would be no way to do anything with the anonymous function, so it would be useless.
I don't see the conceptual difference between a "def-expression" (if it were syntactically possible) and a lambda-expression. What is the difference in your view? Are you sure that difference exists? (It wouldn't be the first time that people ascribe powers to lambda that it doesn't have. :-)
add_callback(button1, def add3(a): return a+3)
Two questions about this example: (1) Do you expect the name 'add3' to be bound in the surrounding scope? (2) What is the purpose of the name other than documenting the obvious? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 3/10/09, Guido van Rossum <guido@python.org> wrote:
'Anonymous' is a different concept from 'expression-defined' despite the tendency to conflate the two.
... Why is it important to you to split this particular hair?
An expression that *creates* and returns a function is useful.
A way to create unnamed functions may or may not be useful.
Right now, the two are tied together, as lambda is the best way to do either. Mentally untangling them might lead to better code.
Yes. The __name__ attribute might never be used, but I personally still prefer that it be meaningful. When someone says they need anonymous functions, I hear: "I really, really *need* the __name__ to be useless!" In the past, I had sometimes just assumed they were wrong. Stephen has given me a glimpse of a mindset which really might need the name to be useless. But neither he nor I normally think that way, so it might still be YAGNI. Terry has pointed out that they may actually mean "I need def to be an expression", with the reference to anonymity being a red herring, because the two concepts are currently confounded.
Moreover, unless you used a decorator, there would be no way to do anything with the anonymous function, so it would be useless.
Thus my troubles seeing the point of the people who care about "anonymous functions."
The only differences *I* see are syntactical warts in lambda. That said, I may be missing something myself, as lambda has had passionate defenders.
add_callback(button1, def add3(a): return a+3)
(1) Do you expect the name 'add3' to be bound in the surrounding scope?
No. But I agree that expectations would differ, so that either would be acceptable, but either would feel like a wart to at least some people.
(2) What is the purpose of the name other than documenting the obvious?
It isn't always quite this obvious. It is more useful in tracebacks. (Though perhaps printing source instead of name would be even better, for functions short enough to be reasonable expressions.) The __name__ is available in case you want to use it for a dispatch table, or to populate fields in an alternative User Interface. (For example, accessibility APIs) -jJ
On Tue, Mar 10, 2009 at 7:44 PM, Jim Jewett <jimjjewett@gmail.com> wrote:
When someone says they need anonymous functions, I hear:
"I really, really *need* the __name__ to be useless!"
I think you need your hearing tuned. By convention anonymous functions and expressions creating functions are almost always synonymous, in almost all languages. So people use the shorter term "anonymous functions" when what they really care about is "expression syntax for creating new functions on the fly".
Well, those have been discussed at length and depth, and nobody has come up with an acceptable syntax to embed a block of statements in the midst of an expression, in Python. That's why they are separate.
When people want the name, they can give it a name using a def statement. I don't accept your argument against that which seems to go along the lines of "but maybe they might want the name later". You can write unreadable code without lambda too. And yes, lambda can be abused; for a really evil example see this recipe: http://code.activestate.com/recipes/148061/ (note the mis-use of the term "one liner" :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Dearest all, Forgive me if I seem frustrated, but it's been a very long day. After catching up on a lot of the discussion here, I am feeling something between astonishment and disappointment. Forgive me as I am new here, but I had expected python-ideas to be filled with remarkable people. Instead I find that many people don't even understand *why* Python is the way it is!! Never mind understanding what it is that they are talking about. It seems as if jumping in with an opinion counts for more than doing some basic research. Forgive me, but the likes of equating expressions with statements or the desire for anonymous functions with some bizarre dislike of the __name__ attribute just shouts out sheer ignorance/stupidity. I had expected more. From everyone. As for the various lambda proposals I've seen, none of them are anywhere near Pythonic!! Some of you, could you please google "lambda site:mail.python.org" and then read for a little while? For some bizarre reason, I had expected those on this list to be Masters of Python. If not why would they care to improve the language? But some of you clearly are not and should spend a lot of time *reading*. And, please, get hold of as many Python libraries as you can and read the code until you feel you truly do get the essence of what is Pythonic. I am really sorry if I've offended anyone. Frustrations aside, I do mean this in a constructive way. If we all individually would apply a little bit more effort, then we'd collectively benefit and have a much better language. Thanks for bearing with me. -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
tav <tav@espians.com> writes:
I'm glad your expectations have been re-adjusted early on. I don't know what would have led you to such an expectation.
Instead I find that many people don't even understand *why* Python is the way it is!!
That's because there are many people who have yet to find these things out. Such people are not excluded from this list.
Again, that state may be unfortunate, but I don't know why you would have expectations that it would be otherwise.
For some bizarre reason, I had expected those on this list to be Masters of Python.
That is rather bizarre. If you find something that led you to expect that, please let us know so the fallacy can be corrected.
If not why would they care to improve the language?
Because they are the Users of Python who are interested in improving the language. -- \ “During my service in the United States Congress, I took the | `\ initiative in creating the Internet.” —Al Gore | _o__) | Ben Finney
Hey Ben and others, Sorry if my previous message came across as being rude. It really wasn't meant so. And it definitely wasn't meant as a personal attack on anyone. I just wish that a little bit of time would be taken before jumping in with comments. As for Python-ideas, I had taken as granted that people posting would have more than a passing interest in language design and the nature of Python. Obviously a false assumption. I apologise for this. I think there is a *lot* of value in many of the ideas that float around. On this list, python-dev, irc channels and even in the blogosphere. The problem is that few of these ideas get the real time and attention that they deserve. And seeing as all of our time is limited. And that the resources for the development of Python itself is limited. Ideally we would focus them more constructively and selectively. But then, this is the internet -- I should stop being an idealist ;p Thanks again for bearing with me. -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
Ben Finney <ben+python@benfinney.id.au> writes:
My message had rather an imperious tone that was not intended. I hasten to note that I'm not claiming any special status for myself with regard to this list, or Python's community. I'm a mere interested party, and my response was not intended to speak authoritatively about How Things Are™. My apologies for any mistaken impressions I might have given. -- \ “Every valuable human being must be a radical and a rebel, for | `\ what he must aim at is to make things better than they are.” | _o__) —Niels Bohr | Ben Finney
On Tue, Mar 10, 2009 at 10:29 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
Don't worry, Tav's wording was rather offensive so strong reactions are understandable. I read your message as a totally fine tit-for-tat reply, and in fact it made me discard my own similar draft. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
(1) I can easily write an expression that returns a function of arbitrary complexity. (2) What I can't easily write is an expression that is arbitrarily complex and returns a function with that complexity. Sorry if that's unclear. My point is that if I know what the complexity is in advance I can write it down. If I can't, do I really want to embed this in the middle of some other function? For example:
Then f('bar', 3) returns a function equivalent to lambda i: i+3, etc. and of course the definition of f can be arbitrarily complicated although that complexity must be planned in advance. I can stick f('bar', 3) anywhere I want a function. What I have yet to hear is an explanation of the advantages of providing (2) since we already have (1) and lambda. --- Bruce
Guido van Rossum writes:
Here's a use-case from Emacs. Various modes have callbacks so that users can customize them. A typical case is that a text-mode hook will turn on auto-fill-mode, which is documented as an example to be done like this: (add-hook 'text-mode-hook (lambda () (auto-fill-mode 1))) where *-mode functions called with nil toggle the mode, positive numbers turn it on, and non-positive numbers turn it off. The add-hook function is supposed to be idempotent: it won't add the same hook function if it is already in the hook. The problem is if you change the lambda and execute that form, the changed lambda is now not identical to the lambda on the hook, so the old version won't be removed. This (add-hook 'text-mode-hook (defun turn-on-auto-fill () (auto-fill-mode 1))) neatly avoids the problem by returning the name of the function, the symbol `turn-on-auto-fill', which is callable and so suitable for hanging on the hook. If you change the definition and execute the above form, add-hook *mutates nothing* (the symbol is already present), but because the hook is indirect through the function's symbol and the defun *is* executed, the definition changes ... which is exactly what you want.[1] AMK's use-case could be post-processed as something like (let ((i 0)) (mapcar (lambda () (let ((name (intern (format "foo-%d-callback" i)))) (define-function name (aref slots i)) (aset slots i name) (setq i (1+ i)) name)) slots)) where slots is the vector of anonymous functions. Providing names in this way costs one indirection per callback invocation in Emacs Lisp, but the benefits in readability of tracebacks are large, especially for compiled code.
AIUI, a def-expression binds a callable to an object, while a lambda expression returns a callable. An anonymous def is just lambda by a different name (and I think the code block proponents agree, based on their willingness to accept syntax using def instead of lambda). I don't see how the kind of thing exemplified above would be useful in Python, and from a parallel reply I just saw, I gather Jim agrees. The point is to show how a function-defining expression can be useful in some contexts. This works in Emacs Lisp because in (setq foo (lambda ...)) tools (including the Lisp interpreter) will not recognize foo as a function identifier although its value is a function, while (defun foo ...) marks foo as a function identifier. But in Python (like Scheme) they're basically the same operation, with a little syntactic sugar. Anything that is based on the separation of variable namespace from function namespace is DOA, right? The renaming mapper is a different issue, I think; it depends on computing object names at runtime (ie, the Lisp `intern' operation), not on separate namespaces. I'm not sure offhand how to do that in Python, or even it it's possible; I've never wanted it. Footnotes: [1] N.B. Of course modern Emacsen define turn-on-auto-fill as a standard function. But this is ugly (because of the single flat namespace of Emacs Lisp), and not all modes have their turn-on, turn-off variants. auto-fill-mode was chosen because (a) the semantics are easy to imagine and (b) the use of lambda in a hook is explained by exactly this example in the Emacs Lisp Manual.
On Tue, Mar 10, 2009 at 10:05 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Got it -- sort of like using assignment in an expression in C, to set a variable and return the valule (but not quite, don't worry :-).
AMK's use-case could be post-processed as something like
I assume you're talking about Andrew Koenig's use case -- ANK is Andrew Kuchling, who AFAIK didn't participate in this thread. :-)
IIUC (my Lisp is very rusty) this just assigns unique names to the functions right? You're saying this to satisfy the people who insist that __name__ is always useful right? But it seems to be marginally useful here since the names don't occur in the source. (?)
Right, and so are separations between value and type namespaces (as other languages use, e.g. C++ and Haskell).
You can assign new values to to f.__name__.
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum writes:
Yes. And no, I don't worry about you, but I do worry about what else may be lurking in a language whose designer(s) chose to return the function definition (rather than the name) from define-function. ;-)
I assume you're talking about Andrew Koenig's use case -- ANK is Andrew Kuchling, who AFAIK didn't participate in this thread. :-)
Oops, my bad. Very sorry to all concerned.
IIUC (my Lisp is very rusty) this just assigns unique names to the functions right?
Yes.
But they are at least cosmetically useful to the runtime system (eg, they will be used in reporting tracebacks -- bytecode in the backtrace is hard to read) and accessible to the user (for redefining a callback on-the-fly). The user can't necessarily access the array of callbacks directly (eg, it might be in C) or conveniently (it may be buried deep in a complex structure). It seems plausible to me that the user is most likely to want to redefine a callback that just blew up, too, and this would give you the necessary "handle" in the backtrace. Also, I haven't thought this through, but use of numbers to differentiate the names was just an easy example. An appropriate naming scheme might make it easy to find a skeleton in the source for the generated callback. Eg, if instead of numbers the identifiers were "foo-abort", "foo-retry", and "foo-fail". I don't know if that would be useful in Andrew's use-case. So, yes, marginal, in the sense that I doubt the use cases are common, but I suspect in a few it could be a great convenience. How useful in Python, I don't know ... Emacs Lisp is full of "seemed like the thing to do at the time" design, so the more handles I have the happier I am.
Terry Reedy writes:
That's unfair. Python has "anonymous blocks" all over the place, since every control structure controls one or more of them. It simply requires that they be forgotten at the next DEDENT. Surely you don't advocate that each of them should get a name! I think this is a difference of cognition. Specifically, people who don't want to name blocks as functions may not abstract processes to signatures as easily, and reify whole processes (including all free identifiers!) as objects more easily, as those who don't think naming is a problem.
Since I routinely use standard names 'f' and 'g' (from math) to name functions whose name I do not care about, I am baffled (and annoyed) by
If the cognition hypothesis is correct, of course you're baffled. You "just don't" think that way, while he really does. The annoyance can probably be relieved by s/a developer/some developers/ here:
(repeated) claims such as "Having to name a one-off function adds additional cognitive overload to a developer." (Tav).
I suspect that "overload" is a pun, here. Your rhetorical question
Golly gee, if one cannot decide on standard one-char name, how can he manage the rest of Python?
has an unexpected answer: in the rest of Python name overloading is carefully controlled and scoped into namespaces. If my cognition hypothesis is correct, then a standard one-character name really does bother/confuse his cognition, where maintaining the whole structure of the block from one use to the next somehow does not. (This baffles me, too!) The question then becomes "can Python become more usable to developers unlike you and me without losing some Pythonicity?" Guido seems to think not (I read him as pessimistic on both grounds: the proposed syntax is neither as useful nor as Pythonic as Tav thinks it is).
Stephen J. Turnbull wrote:
It is unfair to dislike false statements? I think that *that* is unfair ;-)
Surely, I did not. And surely you cannot really think I suggested such. Every expression and every statement or group of statements defines a function on the current namespaces, but I was talking about Python function objects. And I never said that they necessarily should get an individual name (and indeed I went on to say that I too use 'f' and 'g' as stock, don't-care names) but only that I dislike the silly claim that being named '<lambda>' is a virtue. And this was in the context you snipped of Aahz saying that some disliked the *use* of lambda expressions (as opposed to the promotion of their result as superior).
I think this is a difference of cognition.
I do not think it a 'difference of cognition', in the usual sense of the term, to think that a more informative traceback is a teeny bit superior, and certainly not inferior, to a less informative traceback. Unless of course you mean that all disagreements are such. Terry Jan Reedy
On 3/10/09, Terry Reedy <tjreedy@udel.edu> wrote:
Stephen J. Turnbull wrote:
I think this is a difference of cognition.
The question is which traceback will be more informative. A 50-Meg memory dump will be even more informative, but few people will want to sift through it. I *think* at least some of the lambda lovers are saying something akin to: ''' This little piece of logic isn't a worth naming as a section; it is just something that I would do interactively once I got to this point. I don't *want* a debugging pointer right to this line, I *want* to go to the enclosing function to get my bearings. ''' I'm not convinced, because I've seen so many times when a lambda actually is crucial to the bug. That said, I'm the sort of person who will break up and name subunits even if I have to resort to names like fn_XXX_slave_1. And I will certainly admit that there are times when it would be more useful if the traceback showed the source code of the lambda instead of showing <lambda> or skipping the frame. -jJ
Jim Jewett wrote:
I'm not convinced, because I've seen so many times when a lambda actually is crucial to the bug.
I think this is just a special case of a more general problem, that a line number is not always a sufficiently fine-grained piece of information when you're trying to pinpoint an error. You can get the same thing even when lambdas are not involved. It's particularly bad when an expression spans more than one line, because CPython currently doesn't even tell you the line containing the error, but the one where the whole statement started. Ideally, the traceback would show you not just the exact line, but the exact *token* where the error occurred. The technology exists to do this, it's just a matter of deciding to incorporate it into Python. -- Greg
Terry Reedy writes:
Stephen J. Turnbull wrote:
Terry Reedy writes:
No. It is unfair to use factives: the proponents of code blocks don't claim that the uninformative name is a virtue. They claim that the effort required to deal with an unnecessary name is a defect, and so removing that effort is a virtue. You evidently have no answer for that argument, so you reinterpret in precisely the kind of word- twisting way that bothers you when I do it:
That was a rhetorical question, properly marked as such with a exclamation point rather than a question mark. Not to mention being immediately preceded by the obviously correct rationale for the obviously correct answer. How did you miss it? I did have a real point, which was that if the only use of an anonymous block is immediately juxtaposed to that DEDENT, is there really any harm to the lack of the name? In fact, in debugging you have the name of the using function (which should be short and readable, or you're not going to have fun anyway), and a line number, so there is no trouble identifying the problematic code, nor the execution history that led to it. A good debugger might even provide the arguments to the code block as part of the stack trace, which you would have to go to extra effort to get if it were presented merely as a suite. Ie, in this kind of use case a code block could be considered a kind of meta-syntax that tells debuggers "these variables are of interest, so present me, and them, as a pseudo-stack frame". The sticking point, AIUI, is that the code block proponents have not identified a use case where the anonymous block is immediately used, while there is no Pythonic equivalent. So that "harmless" (YMMV) extension is unnecessary, and the extension violates TOOWTDI if that's all it's good for. The real power of "code blocks" (that Python doesn't have) comes when they can be passed around as objects ... but there doesn't seem to be a way to define them so as to exclude the obnoxious uses such as in callbacks, or indeed such a use case other than callbacks. And that violates the Pythonista's sense of good style. So what I want to know (and my question is directed to the code block proponents, not to you) is where is the Pythonic use case? All those Ruby programmers can't be wrong ... can they? Of course they can!<wink> But even so, I'd like to understand what the code block proponents think they're seeing that isn't there, at least not for you and me. *We* (including the BDFL!?) could be wrong, too, maybe there is something special about code blocks that Python could benefit from incorporating. Or maybe there's a better way to teach Python, so that people will use Pythonic idioms instead of reaching for "code blocks."
On Mon, Mar 9, 2009 at 6:04 AM, tav <tav@espians.com> wrote:
Sounds like you might as well write a decorator named @using: @using(employees.select) def _(employee): if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
I like that (which is why I proposed allowing lambda decorators last month), but I'm uncomfortable with how after all is said and done, _ will either be set to something that's not a callable or to a callable that no one is ever supposed to call. Perhaps if we allowed for this: @using(employees.select) as results def (employee): #It is mandatory that no name be used here and #that parentheses are included if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) #results = [ list of employees ] to be syntatic sugar for this: def callback(employee): if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) results = using(employees.select)(callback) #results = [ list of employees ] Similarly, the horrible Java-style callback mentioned earlier in the thread self.Bind(wx.BUTTON, lamda: evt <some huge block of code here> , mybutton) might with a change in API become something like @Bind(wx.BUTTON, mybutton) as connected_button_object def (event): #do event handling stuff… return Ruby blocks let them write,
(1..10).map { |x| 2 * x } => [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
which for us would be a simple [2 * x for x in range(1, 11)], but their version has the advantage of being able to be expanded to a series of expressions and statements instead of a single expression if need be. With my proposal, someone could write:
Basically, the "as" would be used to indicate "no one cares about the following function by itself, they just want to use it as a callback to get some result". My-doomed-proposally-yours, -- Carl
On Mon, Mar 9, 2009 at 10:59 PM, Carl Johnson <cmjohnson.mailinglist@gmail.com> wrote: [Guido]
Well, _ is by convention often used as a "throw-away" result. So I am totally comfortable with this. Or, at least, I am as comfortable with it as I am with writing name, phone, _, _ = record to unpack a 4-tuple when the last two elements are not needed. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
I personally don't mind anonymous functions, I use them when I can fit everything on mostly one line and they don't have any side effects. None of my decorators that I write actually call the function they are passed either. So there could be a lambda statement... @using(employees.select) lambda employee: if employee.developer and employee.not_using_python: fireDeveloper(employee) ...which would make a function that isn't named, purely for its side effects. -$0.02 Joel
On Mon, Mar 9, 2009 at 12:39 PM, Guido van Rossum <guido@python.org> wrote:
What would `using` here do that decorating the temp function with employees.select itself wouldn't do? All you are doing is saying "Pass this function to this function" which is exactly what decorators already do.
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
On Tue, Mar 10, 2009 at 10:13 AM, Calvin Spealman <ironfroggy@gmail.com> wrote:
I think the original proposal was implying some kind of loop over the values returned by employees.select(). But it's really irrelevant for the equivalency. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
tav wrote:
Maybe if you come up with an example that isn't written with already existing python syntax as easy (or even more easily): for employee in employees: # or employees.select() if you like if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) -panzi
On 3/9/2009 2:04 PM, tav wrote:
I believe this is just an extension to the lambda keyword. If lambdas could define a block, not just a statement, this would e.g. be employees.select( lambda employee: if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) ) or tmp = lambda employee: if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) employees.select(tmp) I see no reason for introducing two new keywords to do this, as you are really just enhancing the current lambda keyword. On the other hand, turning blocks into anonymous functions would be very useful for functional programming. As such, I like your suggestion. This also has a great potential for abuse (as in writing unreadable code), just consider how anonymous classes are used in Java's GUI toolkits. I really don't want to see self.Bind(wx.BUTTON, lamda: evt <some huge block of code here> , mybutton) in wxPython code. But Java programmers coming to Pytho would jump to this, as they have been brain washed to use anonymous classes for everything (no pun intended). Sturla Molden
Hey Stuart,
I believe this is just an extension to the lambda keyword. If lambdas could define a block, not just a statement, this would e.g. be
Multi-line lambdas would be nice, but I struggle to find a way to do so in a Pythonic manner. See: http://unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html It would be nice to find a way though...
You might as well just use ``def`` above...
I see no reason for introducing two new keywords to do this, as you are really just enhancing the current lambda keyword.
I agree that two new keywords is a bit much. I tried to re-use ``with`` initially -- but I guess people would be confused by the conflicting semantics.
On the other hand, turning blocks into anonymous functions would be very useful for functional programming. As such, I like your suggestion.
Thanks =) -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
On Mon, Mar 09, 2009, Sturla Molden wrote:
There's a substantial minority (possibly even a majority) in the Python community that abhors functional programming. Even among those who like functional programming, there's a substantial population that dislikes extensive use of anonymous functions. The trick to getting features for functional programming accepted is to make them look as Pythonic as possible. Right now, I'm somewhere between -0 and -1 on this proposal, because all the motivation I see looks like it's perfectly satisfied by using ``def`` instead of lambda. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson
Hey Aahz,
The trick to getting features for functional programming accepted is to make them look as Pythonic as possible.
I spent considerable effort to make the using/do statement as Pythonic as possible. Could you please elaborate on what you don't like about it? Please note that the lambda thing was Sturla's follow up comment... -- Thanks, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
On 3/9/2009 4:53 PM, tav wrote:
If I can elaborate as well. There are three things I don't like: 1. You are introducing two new keywords. Solving problems by constantly adding new syntax is how programming languages are designed in Redmond, WA. I don't exactly know what Pythonic means, but bloating the syntax is not. 2. Most of this is covered by 'def'. Python allows functions to be nested. Python does support closures. 3. Anonymous classes in Java have more cases for abuse than use. Just see how they are abused to write callbacks/handlers. They are a notorious source of unreadable and unmaintainable spaghetti code. S.M.
On Mon, Mar 09, 2009, tav wrote:
That was a general point. The specific point was what you cut: your proposal seems to offer little advantage over a ``def``. You need to justify yourself more thoroughly. Also, because this list is archived, you should probably include your entire argument here rather than referring to an external web page. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson
Hallöchen! tav writes:
Two things for me: The "using ..." is not well-readable. The "(employee)" sits clumsily at the end of the line whithout any connection to the rest. Even worse, the "do" anticipates the ":", which in Python already means "do". And secondly, I'm not comfortable with the fact that the return value is the first (or last? or all?) expression the interpreter stumbles over. Because Python distinguishs between expressions and statements, you have to look twice to see what actually happens in the block. In other words, expressions work differently in these almost-functions, so we end up with two kinds of functions that have different semantic rules. This makes reading more difficult, as well as code-reuse. Tschö, Torsten. -- Torsten Bronger, aquisgrana, europa vetus Jabber ID: torsten.bronger@jabber.rwth-aachen.de
On 3/9/2009 4:45 PM, Aahz wrote:
There's a substantial minority (possibly even a majority) in the Python community that abhors functional programming.
There are a substantial minority that use Python for scientific computing (cf. numpy and scipy, the Hubble space telescope, the NEURON simulator, Sage, etc.) For numerical computing, functional programming often leads to code that are shorter and easier to read. That is, equations look like functions, not like classes. S.M.
On Mon, Mar 09, 2009, Sturla Molden wrote:
Yes, I know; I'm just pointing out that Python is not a pure functional language and that there's a tension within the community about how far Python should go in the functional direction. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson
On Tue, 10 Mar 2009 03:24:36 am Sturla Molden wrote:
I don't understand what you mean. As far as I can see, equations never look like classes in Python, regardless of whether you are using functional programming, object-oriented programming or procedural programming. Can you give me an example of what you mean? Secondly, the proposal relates to *anonymous* functions, which is a small part of functional programming. Perhaps they are necessary in a purely functional programming language, but Python is not such a language. Python never *requires* anonymous functions, they are a convenience and that's all. -- Steven D'Aprano
Aahz wrote:
I am not one of them, if there really are such.
Like many other Pythonistas I recognize that that an uninformative stock name of '<lambda>' is defective relative to an informative name that points back to readable code. What I dislike is the anonymity-cult claim that the defect is a virtue. Since I routinely use standard names 'f' and 'g' (from math) to name functions whose name I do not care about, I am baffled (and annoyed) by (repeated) claims such as "Having to name a one-off function adds additional cognitive overload to a developer." (Tav). Golly gee, if one cannot decide on standard one-char name, how can he manage the rest of Python? (I also, like others, routinely use 'C' for class and 'c' for C instance. What next? A demand for anonymous classes? Whoops, I just learned that Java has those.) But I have no problem with the use of lambda expressions as a convenience, where appropriate. Terry Jan Reedy
On Mon, Mar 9, 2009 at 4:09 PM, Terry Reedy <tjreedy@udel.edu> wrote:
Andrew Koening once gave me a good use case where lambdas are really a lot more convenient than named functions. He was initializing a large data structure that was used by an interpreter for some language. It was a single expression (probably a list of tuples or a dict). Each record contained various bits of information (e.g. the operator symbol and its precedence and associativity) as well as a function (almost always a very simple lambda) that implemented it. Since this table was 100s of records long, it would have been pretty inconvenient to first have to define 100s of small one-line functions and give them names, only to reference them once in the initializer. This use case doesn't have a nice equivalent without anonymous functions (though I'm sure that if there really was no other way it could be done, e.g. using registration=style decorators). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
With a small trick you don't need lambdas for this, if the keys are python identifiers. If they aren't you can add the real key to the info tuple and then generate a new dict in a oneliner. But yeah, it's a bit ugly/abusive. But it is possible without defining parts of the tuple at different places! :) def info(*args): def wrapper(f): return args + (f,) return wrapper def mkdict(): @info("a",12,9.9,None) def foo(): print "this is foo" @info("b",23,0.0,foo) def bar(): print "this is bar" @info("c",42,3.1415,None) def baz(): print "this is baz" return locals() print mkdict() -panzi
Guido van Rossum wrote:
Initializing such structures is one of the use cases I intended under 'where appropriate'. Adding more powerful expressions, like comprehensions (and g.e's) that do not break Python's basic syntactic model of mixed expressions and indented statements has added to the convenience.
The convenience is from having function expressions. If the expression syntax allowed the optional attachment of a name, it would be just as convenient. In some cases, I am sure people would find it even more convenient if they could add in a name, especially when there is nothing else in the structure to serve as a substitute. 'Anonymous' is a different concept from 'expression-defined' despite the tendency to conflate the two. Terry Jan Reedy
On Tue, Mar 10, 2009 at 1:15 PM, Terry Reedy <tjreedy@udel.edu> wrote:
If I read you correctly you're saying that having an expression that returns a function (other than referencing it by name) is not the same as having anonymous functions. This sounds like quite the hairsplitting argument. Why is it important to you to split this particular hair? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 3/10/09, Guido van Rossum <guido@python.org> wrote:
On Tue, Mar 10, 2009 at 1:15 PM, Terry Reedy <tjreedy@udel.edu> wrote:
On Mon, Mar 9, 2009 at 4:09 PM, Terry Reedy <tjreedy@udel.edu> wrote:
... What I dislike is the anonymity-cult claim that the defect is a virtue.
'Anonymous' is a different concept from 'expression-defined' despite the tendency to conflate the two.
An expression that *creates* and returns a function is useful. A way to create unnamed functions may or may not be useful. Right now, the two are tied together, as lambda is the best way to do either. Mentally untangling them might lead to better code. If the name in a def were optional, it would meet the perceived need for anonymity, but still wouldn't meet the need for creating and returning a function within a single expression. # Would this really ever be useful? # Not to me, but the anon-lovers suggest yes. # Cognition difference, or just confounding the two uses of lambda? def (a): return a+3 On the other hand, if def became an expression, it would meet the need for function-creating expressions (and would have at least reduced the need for decorators). add_callback(button1, def add3(a): return a+3) (And yes, I understand that there are reasons why class, def, and import do not return values, even if I sometimes wish they did.) -jJ
On Tue, Mar 10, 2009 at 3:22 PM, Jim Jewett <jimjjewett@gmail.com> wrote:
I'm feeling really dense right now -- I still don't see the difference between the two. Are you saying that you would prefer an expression that creates a *named* function? That seems to be really bizarre -- like claiming that you don't like expressions that return anonymous numbers.
Moreover, unless you used a decorator, there would be no way to do anything with the anonymous function, so it would be useless.
I don't see the conceptual difference between a "def-expression" (if it were syntactically possible) and a lambda-expression. What is the difference in your view? Are you sure that difference exists? (It wouldn't be the first time that people ascribe powers to lambda that it doesn't have. :-)
add_callback(button1, def add3(a): return a+3)
Two questions about this example: (1) Do you expect the name 'add3' to be bound in the surrounding scope? (2) What is the purpose of the name other than documenting the obvious? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
On 3/10/09, Guido van Rossum <guido@python.org> wrote:
'Anonymous' is a different concept from 'expression-defined' despite the tendency to conflate the two.
... Why is it important to you to split this particular hair?
An expression that *creates* and returns a function is useful.
A way to create unnamed functions may or may not be useful.
Right now, the two are tied together, as lambda is the best way to do either. Mentally untangling them might lead to better code.
Yes. The __name__ attribute might never be used, but I personally still prefer that it be meaningful. When someone says they need anonymous functions, I hear: "I really, really *need* the __name__ to be useless!" In the past, I had sometimes just assumed they were wrong. Stephen has given me a glimpse of a mindset which really might need the name to be useless. But neither he nor I normally think that way, so it might still be YAGNI. Terry has pointed out that they may actually mean "I need def to be an expression", with the reference to anonymity being a red herring, because the two concepts are currently confounded.
Moreover, unless you used a decorator, there would be no way to do anything with the anonymous function, so it would be useless.
Thus my troubles seeing the point of the people who care about "anonymous functions."
The only differences *I* see are syntactical warts in lambda. That said, I may be missing something myself, as lambda has had passionate defenders.
add_callback(button1, def add3(a): return a+3)
(1) Do you expect the name 'add3' to be bound in the surrounding scope?
No. But I agree that expectations would differ, so that either would be acceptable, but either would feel like a wart to at least some people.
(2) What is the purpose of the name other than documenting the obvious?
It isn't always quite this obvious. It is more useful in tracebacks. (Though perhaps printing source instead of name would be even better, for functions short enough to be reasonable expressions.) The __name__ is available in case you want to use it for a dispatch table, or to populate fields in an alternative User Interface. (For example, accessibility APIs) -jJ
On Tue, Mar 10, 2009 at 7:44 PM, Jim Jewett <jimjjewett@gmail.com> wrote:
When someone says they need anonymous functions, I hear:
"I really, really *need* the __name__ to be useless!"
I think you need your hearing tuned. By convention anonymous functions and expressions creating functions are almost always synonymous, in almost all languages. So people use the shorter term "anonymous functions" when what they really care about is "expression syntax for creating new functions on the fly".
Well, those have been discussed at length and depth, and nobody has come up with an acceptable syntax to embed a block of statements in the midst of an expression, in Python. That's why they are separate.
When people want the name, they can give it a name using a def statement. I don't accept your argument against that which seems to go along the lines of "but maybe they might want the name later". You can write unreadable code without lambda too. And yes, lambda can be abused; for a really evil example see this recipe: http://code.activestate.com/recipes/148061/ (note the mis-use of the term "one liner" :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Dearest all, Forgive me if I seem frustrated, but it's been a very long day. After catching up on a lot of the discussion here, I am feeling something between astonishment and disappointment. Forgive me as I am new here, but I had expected python-ideas to be filled with remarkable people. Instead I find that many people don't even understand *why* Python is the way it is!! Never mind understanding what it is that they are talking about. It seems as if jumping in with an opinion counts for more than doing some basic research. Forgive me, but the likes of equating expressions with statements or the desire for anonymous functions with some bizarre dislike of the __name__ attribute just shouts out sheer ignorance/stupidity. I had expected more. From everyone. As for the various lambda proposals I've seen, none of them are anywhere near Pythonic!! Some of you, could you please google "lambda site:mail.python.org" and then read for a little while? For some bizarre reason, I had expected those on this list to be Masters of Python. If not why would they care to improve the language? But some of you clearly are not and should spend a lot of time *reading*. And, please, get hold of as many Python libraries as you can and read the code until you feel you truly do get the essence of what is Pythonic. I am really sorry if I've offended anyone. Frustrations aside, I do mean this in a constructive way. If we all individually would apply a little bit more effort, then we'd collectively benefit and have a much better language. Thanks for bearing with me. -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
tav <tav@espians.com> writes:
I'm glad your expectations have been re-adjusted early on. I don't know what would have led you to such an expectation.
Instead I find that many people don't even understand *why* Python is the way it is!!
That's because there are many people who have yet to find these things out. Such people are not excluded from this list.
Again, that state may be unfortunate, but I don't know why you would have expectations that it would be otherwise.
For some bizarre reason, I had expected those on this list to be Masters of Python.
That is rather bizarre. If you find something that led you to expect that, please let us know so the fallacy can be corrected.
If not why would they care to improve the language?
Because they are the Users of Python who are interested in improving the language. -- \ “During my service in the United States Congress, I took the | `\ initiative in creating the Internet.” —Al Gore | _o__) | Ben Finney
Hey Ben and others, Sorry if my previous message came across as being rude. It really wasn't meant so. And it definitely wasn't meant as a personal attack on anyone. I just wish that a little bit of time would be taken before jumping in with comments. As for Python-ideas, I had taken as granted that people posting would have more than a passing interest in language design and the nature of Python. Obviously a false assumption. I apologise for this. I think there is a *lot* of value in many of the ideas that float around. On this list, python-dev, irc channels and even in the blogosphere. The problem is that few of these ideas get the real time and attention that they deserve. And seeing as all of our time is limited. And that the resources for the development of Python itself is limited. Ideally we would focus them more constructively and selectively. But then, this is the internet -- I should stop being an idealist ;p Thanks again for bearing with me. -- love, tav plex:espians/tav | tav@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian
Ben Finney <ben+python@benfinney.id.au> writes:
My message had rather an imperious tone that was not intended. I hasten to note that I'm not claiming any special status for myself with regard to this list, or Python's community. I'm a mere interested party, and my response was not intended to speak authoritatively about How Things Are™. My apologies for any mistaken impressions I might have given. -- \ “Every valuable human being must be a radical and a rebel, for | `\ what he must aim at is to make things better than they are.” | _o__) —Niels Bohr | Ben Finney
On Tue, Mar 10, 2009 at 10:29 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
Don't worry, Tav's wording was rather offensive so strong reactions are understandable. I read your message as a totally fine tit-for-tat reply, and in fact it made me discard my own similar draft. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
(1) I can easily write an expression that returns a function of arbitrary complexity. (2) What I can't easily write is an expression that is arbitrarily complex and returns a function with that complexity. Sorry if that's unclear. My point is that if I know what the complexity is in advance I can write it down. If I can't, do I really want to embed this in the middle of some other function? For example:
Then f('bar', 3) returns a function equivalent to lambda i: i+3, etc. and of course the definition of f can be arbitrarily complicated although that complexity must be planned in advance. I can stick f('bar', 3) anywhere I want a function. What I have yet to hear is an explanation of the advantages of providing (2) since we already have (1) and lambda. --- Bruce
Guido van Rossum writes:
Here's a use-case from Emacs. Various modes have callbacks so that users can customize them. A typical case is that a text-mode hook will turn on auto-fill-mode, which is documented as an example to be done like this: (add-hook 'text-mode-hook (lambda () (auto-fill-mode 1))) where *-mode functions called with nil toggle the mode, positive numbers turn it on, and non-positive numbers turn it off. The add-hook function is supposed to be idempotent: it won't add the same hook function if it is already in the hook. The problem is if you change the lambda and execute that form, the changed lambda is now not identical to the lambda on the hook, so the old version won't be removed. This (add-hook 'text-mode-hook (defun turn-on-auto-fill () (auto-fill-mode 1))) neatly avoids the problem by returning the name of the function, the symbol `turn-on-auto-fill', which is callable and so suitable for hanging on the hook. If you change the definition and execute the above form, add-hook *mutates nothing* (the symbol is already present), but because the hook is indirect through the function's symbol and the defun *is* executed, the definition changes ... which is exactly what you want.[1] AMK's use-case could be post-processed as something like (let ((i 0)) (mapcar (lambda () (let ((name (intern (format "foo-%d-callback" i)))) (define-function name (aref slots i)) (aset slots i name) (setq i (1+ i)) name)) slots)) where slots is the vector of anonymous functions. Providing names in this way costs one indirection per callback invocation in Emacs Lisp, but the benefits in readability of tracebacks are large, especially for compiled code.
AIUI, a def-expression binds a callable to an object, while a lambda expression returns a callable. An anonymous def is just lambda by a different name (and I think the code block proponents agree, based on their willingness to accept syntax using def instead of lambda). I don't see how the kind of thing exemplified above would be useful in Python, and from a parallel reply I just saw, I gather Jim agrees. The point is to show how a function-defining expression can be useful in some contexts. This works in Emacs Lisp because in (setq foo (lambda ...)) tools (including the Lisp interpreter) will not recognize foo as a function identifier although its value is a function, while (defun foo ...) marks foo as a function identifier. But in Python (like Scheme) they're basically the same operation, with a little syntactic sugar. Anything that is based on the separation of variable namespace from function namespace is DOA, right? The renaming mapper is a different issue, I think; it depends on computing object names at runtime (ie, the Lisp `intern' operation), not on separate namespaces. I'm not sure offhand how to do that in Python, or even it it's possible; I've never wanted it. Footnotes: [1] N.B. Of course modern Emacsen define turn-on-auto-fill as a standard function. But this is ugly (because of the single flat namespace of Emacs Lisp), and not all modes have their turn-on, turn-off variants. auto-fill-mode was chosen because (a) the semantics are easy to imagine and (b) the use of lambda in a hook is explained by exactly this example in the Emacs Lisp Manual.
On Tue, Mar 10, 2009 at 10:05 PM, Stephen J. Turnbull <stephen@xemacs.org> wrote:
Got it -- sort of like using assignment in an expression in C, to set a variable and return the valule (but not quite, don't worry :-).
AMK's use-case could be post-processed as something like
I assume you're talking about Andrew Koenig's use case -- ANK is Andrew Kuchling, who AFAIK didn't participate in this thread. :-)
IIUC (my Lisp is very rusty) this just assigns unique names to the functions right? You're saying this to satisfy the people who insist that __name__ is always useful right? But it seems to be marginally useful here since the names don't occur in the source. (?)
Right, and so are separations between value and type namespaces (as other languages use, e.g. C++ and Haskell).
You can assign new values to to f.__name__.
-- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum writes:
Yes. And no, I don't worry about you, but I do worry about what else may be lurking in a language whose designer(s) chose to return the function definition (rather than the name) from define-function. ;-)
I assume you're talking about Andrew Koenig's use case -- ANK is Andrew Kuchling, who AFAIK didn't participate in this thread. :-)
Oops, my bad. Very sorry to all concerned.
IIUC (my Lisp is very rusty) this just assigns unique names to the functions right?
Yes.
But they are at least cosmetically useful to the runtime system (eg, they will be used in reporting tracebacks -- bytecode in the backtrace is hard to read) and accessible to the user (for redefining a callback on-the-fly). The user can't necessarily access the array of callbacks directly (eg, it might be in C) or conveniently (it may be buried deep in a complex structure). It seems plausible to me that the user is most likely to want to redefine a callback that just blew up, too, and this would give you the necessary "handle" in the backtrace. Also, I haven't thought this through, but use of numbers to differentiate the names was just an easy example. An appropriate naming scheme might make it easy to find a skeleton in the source for the generated callback. Eg, if instead of numbers the identifiers were "foo-abort", "foo-retry", and "foo-fail". I don't know if that would be useful in Andrew's use-case. So, yes, marginal, in the sense that I doubt the use cases are common, but I suspect in a few it could be a great convenience. How useful in Python, I don't know ... Emacs Lisp is full of "seemed like the thing to do at the time" design, so the more handles I have the happier I am.
Terry Reedy writes:
That's unfair. Python has "anonymous blocks" all over the place, since every control structure controls one or more of them. It simply requires that they be forgotten at the next DEDENT. Surely you don't advocate that each of them should get a name! I think this is a difference of cognition. Specifically, people who don't want to name blocks as functions may not abstract processes to signatures as easily, and reify whole processes (including all free identifiers!) as objects more easily, as those who don't think naming is a problem.
Since I routinely use standard names 'f' and 'g' (from math) to name functions whose name I do not care about, I am baffled (and annoyed) by
If the cognition hypothesis is correct, of course you're baffled. You "just don't" think that way, while he really does. The annoyance can probably be relieved by s/a developer/some developers/ here:
(repeated) claims such as "Having to name a one-off function adds additional cognitive overload to a developer." (Tav).
I suspect that "overload" is a pun, here. Your rhetorical question
Golly gee, if one cannot decide on standard one-char name, how can he manage the rest of Python?
has an unexpected answer: in the rest of Python name overloading is carefully controlled and scoped into namespaces. If my cognition hypothesis is correct, then a standard one-character name really does bother/confuse his cognition, where maintaining the whole structure of the block from one use to the next somehow does not. (This baffles me, too!) The question then becomes "can Python become more usable to developers unlike you and me without losing some Pythonicity?" Guido seems to think not (I read him as pessimistic on both grounds: the proposed syntax is neither as useful nor as Pythonic as Tav thinks it is).
Stephen J. Turnbull wrote:
It is unfair to dislike false statements? I think that *that* is unfair ;-)
Surely, I did not. And surely you cannot really think I suggested such. Every expression and every statement or group of statements defines a function on the current namespaces, but I was talking about Python function objects. And I never said that they necessarily should get an individual name (and indeed I went on to say that I too use 'f' and 'g' as stock, don't-care names) but only that I dislike the silly claim that being named '<lambda>' is a virtue. And this was in the context you snipped of Aahz saying that some disliked the *use* of lambda expressions (as opposed to the promotion of their result as superior).
I think this is a difference of cognition.
I do not think it a 'difference of cognition', in the usual sense of the term, to think that a more informative traceback is a teeny bit superior, and certainly not inferior, to a less informative traceback. Unless of course you mean that all disagreements are such. Terry Jan Reedy
On 3/10/09, Terry Reedy <tjreedy@udel.edu> wrote:
Stephen J. Turnbull wrote:
I think this is a difference of cognition.
The question is which traceback will be more informative. A 50-Meg memory dump will be even more informative, but few people will want to sift through it. I *think* at least some of the lambda lovers are saying something akin to: ''' This little piece of logic isn't a worth naming as a section; it is just something that I would do interactively once I got to this point. I don't *want* a debugging pointer right to this line, I *want* to go to the enclosing function to get my bearings. ''' I'm not convinced, because I've seen so many times when a lambda actually is crucial to the bug. That said, I'm the sort of person who will break up and name subunits even if I have to resort to names like fn_XXX_slave_1. And I will certainly admit that there are times when it would be more useful if the traceback showed the source code of the lambda instead of showing <lambda> or skipping the frame. -jJ
Jim Jewett wrote:
I'm not convinced, because I've seen so many times when a lambda actually is crucial to the bug.
I think this is just a special case of a more general problem, that a line number is not always a sufficiently fine-grained piece of information when you're trying to pinpoint an error. You can get the same thing even when lambdas are not involved. It's particularly bad when an expression spans more than one line, because CPython currently doesn't even tell you the line containing the error, but the one where the whole statement started. Ideally, the traceback would show you not just the exact line, but the exact *token* where the error occurred. The technology exists to do this, it's just a matter of deciding to incorporate it into Python. -- Greg
Terry Reedy writes:
Stephen J. Turnbull wrote:
Terry Reedy writes:
No. It is unfair to use factives: the proponents of code blocks don't claim that the uninformative name is a virtue. They claim that the effort required to deal with an unnecessary name is a defect, and so removing that effort is a virtue. You evidently have no answer for that argument, so you reinterpret in precisely the kind of word- twisting way that bothers you when I do it:
That was a rhetorical question, properly marked as such with a exclamation point rather than a question mark. Not to mention being immediately preceded by the obviously correct rationale for the obviously correct answer. How did you miss it? I did have a real point, which was that if the only use of an anonymous block is immediately juxtaposed to that DEDENT, is there really any harm to the lack of the name? In fact, in debugging you have the name of the using function (which should be short and readable, or you're not going to have fun anyway), and a line number, so there is no trouble identifying the problematic code, nor the execution history that led to it. A good debugger might even provide the arguments to the code block as part of the stack trace, which you would have to go to extra effort to get if it were presented merely as a suite. Ie, in this kind of use case a code block could be considered a kind of meta-syntax that tells debuggers "these variables are of interest, so present me, and them, as a pseudo-stack frame". The sticking point, AIUI, is that the code block proponents have not identified a use case where the anonymous block is immediately used, while there is no Pythonic equivalent. So that "harmless" (YMMV) extension is unnecessary, and the extension violates TOOWTDI if that's all it's good for. The real power of "code blocks" (that Python doesn't have) comes when they can be passed around as objects ... but there doesn't seem to be a way to define them so as to exclude the obnoxious uses such as in callbacks, or indeed such a use case other than callbacks. And that violates the Pythonista's sense of good style. So what I want to know (and my question is directed to the code block proponents, not to you) is where is the Pythonic use case? All those Ruby programmers can't be wrong ... can they? Of course they can!<wink> But even so, I'd like to understand what the code block proponents think they're seeing that isn't there, at least not for you and me. *We* (including the BDFL!?) could be wrong, too, maybe there is something special about code blocks that Python could benefit from incorporating. Or maybe there's a better way to teach Python, so that people will use Pythonic idioms instead of reaching for "code blocks."
On Mon, Mar 9, 2009 at 6:04 AM, tav <tav@espians.com> wrote:
Sounds like you might as well write a decorator named @using: @using(employees.select) def _(employee): if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
I like that (which is why I proposed allowing lambda decorators last month), but I'm uncomfortable with how after all is said and done, _ will either be set to something that's not a callable or to a callable that no one is ever supposed to call. Perhaps if we allowed for this: @using(employees.select) as results def (employee): #It is mandatory that no name be used here and #that parentheses are included if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) #results = [ list of employees ] to be syntatic sugar for this: def callback(employee): if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) results = using(employees.select)(callback) #results = [ list of employees ] Similarly, the horrible Java-style callback mentioned earlier in the thread self.Bind(wx.BUTTON, lamda: evt <some huge block of code here> , mybutton) might with a change in API become something like @Bind(wx.BUTTON, mybutton) as connected_button_object def (event): #do event handling stuff… return Ruby blocks let them write,
(1..10).map { |x| 2 * x } => [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
which for us would be a simple [2 * x for x in range(1, 11)], but their version has the advantage of being able to be expanded to a series of expressions and statements instead of a single expression if need be. With my proposal, someone could write:
Basically, the "as" would be used to indicate "no one cares about the following function by itself, they just want to use it as a callback to get some result". My-doomed-proposally-yours, -- Carl
On Mon, Mar 9, 2009 at 10:59 PM, Carl Johnson <cmjohnson.mailinglist@gmail.com> wrote: [Guido]
Well, _ is by convention often used as a "throw-away" result. So I am totally comfortable with this. Or, at least, I am as comfortable with it as I am with writing name, phone, _, _ = record to unpack a 4-tuple when the last two elements are not needed. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
I personally don't mind anonymous functions, I use them when I can fit everything on mostly one line and they don't have any side effects. None of my decorators that I write actually call the function they are passed either. So there could be a lambda statement... @using(employees.select) lambda employee: if employee.developer and employee.not_using_python: fireDeveloper(employee) ...which would make a function that isn't named, purely for its side effects. -$0.02 Joel
On Mon, Mar 9, 2009 at 12:39 PM, Guido van Rossum <guido@python.org> wrote:
What would `using` here do that decorating the temp function with employees.select itself wouldn't do? All you are doing is saying "Pass this function to this function" which is exactly what decorators already do.
-- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
On Tue, Mar 10, 2009 at 10:13 AM, Calvin Spealman <ironfroggy@gmail.com> wrote:
I think the original proposal was implying some kind of loop over the values returned by employees.select(). But it's really irrelevant for the equivalency. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
tav wrote:
Maybe if you come up with an example that isn't written with already existing python syntax as easy (or even more easily): for employee in employees: # or employees.select() if you like if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) -panzi
participants (17)
-
Aahz -
Ben Finney -
Bruce Leban -
Calvin Spealman -
Carl Johnson -
Greg Ewing -
Guido van Rossum -
Jan Claeys -
Jim Jewett -
Joel Bender -
Mathias Panzenböck -
Stephen J. Turnbull -
Steven D'Aprano -
Sturla Molden -
tav -
Terry Reedy -
Torsten Bronger