And now for something completely different

Okay, for anyone who's still willing to bear with me, I have a completely different approach that might be more palatable (and perhaps even appealing) to some people. I was reading the documentation for Io ( http://www.iolanguage.com ), particulary about how Io implements control structures: in short, it doesn't. What it's able to do is let the user define control structures in Io itself. For example "if" and "for" are merely functions: for ( i, 1, 10, i println ) if ( b == 0, c + 1, d ) Now, the reason this works in Io (and not in Python) is because while you *can* implement a function that mimics the logic of, say, an if-statement, there's no way to implement lazy evaluation of the resulting values. So for instance: def IF ( condition, iftrue, iffalse ): if condition: return iftrue return iffalse result = IF ( hasattr ( spam, 'eggs' ), spam.eggs, 'not found' ) The expression spam.eggs is evaluated regardless of the truthiness of the condition (in fact, causes an exception), so this construct is basically worthless. So... now, here's my shiny new idea: lazy evaluation of function arguments (by some explicit syntactical indicator). Arguments would not be evaluated until they are encountered *at runtime* in the function body. They could be compiled, but not evaluated. You can accomplish this to some degree by using lambda, but I'd much prefer something indicated in the function signature than in the calling code. That is rather than: def foo ( arg ): return arg ( ) foo ( lambda: x ) I'd prefer to write def foo ( lambda: arg ): return arg # evaluation happens here foo ( x ) ("lambda" not necessarily being the actual name of the token in the second case). I think this would have appeal outside of my particular desire (lazy evaluation is a useful feature in-and-of itself) but would also solve my particular issue (I could implement functional versions of any control structure I like, and even implement new ones). Thoughts? Regards, Cliff

On Wed, Sep 17, 2008 at 3:41 PM, Cliff Wells <cliff@develix.com> wrote:
I'd rather use the time machine. You can already do this quite easily. def foo (arg): return arg() # evaluation happens here foo(lambda: x) or def IF(c,t,f=lambda:None): if c: return t() else: return f() IF(hasattr(spam, 'eggs'), lambda: spam.eggs, lambda: 'not found') The idea of something automagically changing from an unevaluated to an evaluated value strikes me as a very bad one so requiring explicit evaluation here is a *good* thing. Aside from that, why should arg be evaluated on return as opposed to being passed along unevaluated until its value is really needed? How would I check to see if an arg was provided? I certainly can't use a default value of None and then check for None since the check would cause it to be evaluated. Maybe we have a special operator that allows us to look at the unevaluated value. What's the context of the evaluation? Can it access variables in the place where it's evaluated? Maybe there's a special way to do that too. Let's not go down that path. --- Bruce

On Wed, 2008-09-17 at 16:18 -0700, Bruce Leban wrote:
Except I'd prefer not having lambda sprinkled throughout the calling code (or worse, forgotten).
Didn't suggest that. That was merely an example. What I'm suggesting is that the *first* time it's used, evaluate then (my example used return for simplicity).
I don't consider that a problem. You either care about the value of an argument or you don't. If you care then you will eventually evaluate it anyway. If you don't, then don't evaluate it, ever.
Agreed we shouldn't go down that path, but don't agree that it's needed. Cliff

On Wed, Sep 17, 2008 at 5:27 PM, Cliff Wells <cliff@develix.com> wrote:
Back when you were talking about statements -> expressions, I was going to point out deferred function calls, but I also ran into the "lambda is ugly" problem. If it weren't for the fact that backticks ` are difficult to distinguish from regular single quotes ', never mind being a PITA to type on some keyboards, I would *almost* suggest that backticks should be used for expressing "a function without arguments, aka a deferred execution expression". But then I'd have to give myself a -1, as it violates TOOWTDI. Then again, the argument could be made that lambdas are just a long spelling of a def without a name, and that ... add = def (arg1, arg2): arg1+arg2 would be better. At that point, we come to def:'deferred' . And I think that's ever worse than backticks. I don't know of a spelling of deferred execution blocks that I would find reasonable :/ . - Josiah

Josiah Carlson wrote:
I don't know of a spelling of deferred execution blocks that I would find reasonable :/ .
His request is worse than that. He didn't ask for deferred execution, he asked for deferred values. In python Right Now, you can already do deferred execution blocks (forgetting the "I don't like seeing lambdas!" argument). Rather, he wants to be able to have arbitrary expressions be treated as lazy, not at the call-site either but determined by the receiver: """ I'd prefer to write def foo ( lambda: arg ): return arg # evaluation happens here foo ( x ) """ I'm sorry but this will never, ever, ever happen in Python. That is such a *huge* shift in semantics that you should go use a different language. In such languages, it is *impossible* to know when the expression will actually be evaluated, and while this is fine in a purely functional environment, this is not easy to deal with in imperative programming. If you write "foo(bar())", then you have no way of knowing whether "bar()" is ever run. And in a world that depends on side-effects, that forces you to force evaluations of expressions. You would be forced to write "x = bar(); foo(x)" (assuming assignment isn't lazy), which is less clear than having to write "foo(lambda: bar())" in the opposite case. Explicit is better than implicit. -Scott -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu

On Wed, Sep 17, 2008 at 8:05 PM, Scott Dial <scott+python-ideas@scottdial.com> wrote:
What I was thinking (completely unreasonable, I would be -1 on this) is that the following two lines would be equivalent... x = lambda: foo.bar x = `foo.bar` Sadly, that doesn't really gain you anything - though it would be convenient in the case of an If function (or similar): def If(condition, truev, falsev=`None`): if condition: return truev() return falsev() result = If(x < 5, `foo.smallx(x)`, `foo.largex(x)`) Though, because it's still a function call, it's still slow. And even worse, uses a previously decided-upon ugly pair of characters. While I understand that Cliff wanted auto-execution, I think that requiring a call isn't out of the question (what do Ruby thunks require?) - Josiah

On Wed, 2008-09-17 at 16:18 -0700, Bruce Leban wrote:
To be clear, the problem is that this leads to brittle code (I want to *enforce* lazy evaluation in a certain context): IF(a<100, b**a, 0) Oops, forgot lambda and nothing crashes but we now constantly evaluate unbounded values.
Sorry, missed answering this: the context would be unchanged. It would be evaluated as if it were evaluated at the time of the function call. This may be a serious implementation problem, but I'll let the experts answer that. Regards, Cliff

Cliff Wells wrote:
Unless some clever implementation trick could be found, this would have extremely serious performance implications in CPython. Every expression used as a function argument would have to be compiled as a separate function. A closure would need to be created every time it was used, and a Python call executed every time it was evaluated, both fairly expensive operations. Any variables it uses from the surrounding scope would need to be allocated in cells, meaning more memory allocations and access overhead. I could imagine this making the execution of *all* Python code several times slower or worse. -- Greg

Cliff Wells wrote:
You can accomplish this to some degree by using lambda,
You can do it completely as far as I know.
but I'd much prefer something indicated in the function signature than in the calling code.
Given Python's nature as a compiled dynamic language, indications are really needed in both caller and callee, as at present: pass a function and call it. Without an explicit indication at the call site, the compiler would have to compile both direct evaluation and function object creation and the code to select between them at runtime according to an attribute of the function resulting from evaluation of the presumed function expression. This would slow *all* function calls, which are slow enough already. Also, one could not tell what objects get passed to a function without knowing the signature in detail, making code harder to read. There is already a way to be flexible without magical arg object switching and with only exceptional need for del/lambda. def iff(cond, if_result, else_result): if cond: if hasattr(if_result, '__call__'): return if_result() else: return if_result else: if hasattr(else_result, '__call__'): return else_result() else: return else_result for i in [0,1,2]: print("I gave you %s gold piece%s." % (i,iff(i==1, '', 's'))) #3.0 #prints (imagine a typical RPG game) I gave you 0 gold pieces. I gave you 1 gold piece. I gave you 2 gold pieces. One only needs the wrapper if the object to be returned is a function example -- iff(cond, lambda: math.sin, lambda: math.cos)(x) or if the expression needs to be guarded by the condition because it would otherwise be invalid (iff(x, lambda x=x: sin(x)/x, 1) -- this might work without the default arg, but might have scope problems) or if it would take a long time. But many conditional values are constant or quick to evaluate and the short-circuiting is not needed. Many of the exceptions might better be defined in functions anyway. def sin_or_cos(cond,x): if cond: return math.sin(x) else: return math.cos(x) def sin_over_x(x): if x: return math.sin(x)/x else: return 0 Terry J. Reedy

On Wed, 2008-09-17 at 23:54 -0400, Terry Reedy wrote:
Yes, I had this suspicion, but I figured I'd wait for an authority to make it plain. However, consider this: foo ( a + b ) # a + b is compiled but not currently evaluated Now, clearly Python would need to know at runtime if foo() accepts a lazy argument prior to calling it, however I'm reminded that Python 3.0 is adding optional static type checking (I believe) which seems to bring the same issue to the table (and possibly at the same expense you mention). Would the mechanism not be more or less the same? Check signature and then apply arguments? Cliff

On Wed, 2008-09-17 at 23:54 -0400, Terry Reedy wrote:
There's one other approach: have two types of functions: <function> and <function-with-lazy-args>. Whenever Python encounters a declaration of a function with a lazy argument it creates the latter. This would allow Python to know at runtime how to handle it. The difference could be transparent to user code (although it might be unappealing to disguise such a difference). Cliff

On Thu, Sep 18, 2008 at 7:01 AM, Cliff Wells <cliff@develix.com> wrote:
Why don't you just use strings instead of actual arguments and then just evaluate them if you need them? Someone has already suggested that... As I briefly understand your idea of making Python a more functional language (I like FP myself), I don't think Python is ever going to be one. If you want some FP, just use another language (lazy args will not be implemented ever for sure) Or you can fork Python :) Cheers, Peter

Cliff Wells wrote:
This is what C and languages sort of have with functions and text macros. Without a name convention (such as CPython uses in its codebase, with macros ALL_CAPS, I believe), one cannot tell, for instance, whether f(x) will or will not result in the passing of 'x' or eval(x). It is what some Lisps used to have -- functions and special functions. Users had to memorized which args of which functions were normal and lazy. There must have been some reason why that was superceded. But to repeat: with Python's dynamic name binding, the compiler cannot in general tell whether the 'f' in 'f(x)' will be bound at runtime to a regular or special function, so as I said above, it would have to code for both possibilities.
This was already assumed in my original 'select between them at runtime according to an attribute of the function'.
Unless the argument expression has visible side-effects when evaluated. I think there are also namespace issues that you have missed. Consider def f(`a,`b): # ` indicates lazy evaulation, after Lisp's 'quote ..... blaf + f(a+b,c) -3 When f evaluates a bound to 'a+b', it has to *not* resolve 'b' to f's 'b' but to the caller's 'b'. As someone else said, the unevaluated code for 'a+b' is not enough. A full enclosing function is needed. tjr

On Thu, 2008-09-18 at 16:10 -0400, Terry Reedy wrote:
Cliff Wells wrote:
Sorry, I must have missed that in reading your reply, but yes both would work. Which of them would be preferable is an implementation detail outside my scope of knowledge.
I think this is the stickiest point (from a philosophical perspective anyway). It's true an explicit side-effect might not happen. However, we already have the inverse of this problem (simple expressions disguising side-effects). Consider: a + b Given that a and be might not be numeric types, what does the above mean? Someone mentioned earlier (sorry, lost the reference) that they found the idea of having to refer to a function signature to unappealing, but magic methods already present a similar problem (disguising potential side-effects). We depend on sane library authors to not abuse this feature. In any case, whenever encountering such a construct we are forced to either assume that the class author did the right thing and the operation makes sense or we refer to the documentation/source to decipher the meaning. I think what I'm suggesting isn't so dissimilar. I will admit that the mere *presence* of a class implies that such side-effects might be present whereas functions only appear ordinary, so there is a tiny bit of fair warning. Still, when encountering something like iff ( cond, iftrue, iffalse ) in a library, I think what I'm suggesting is the least surprising result. It comes down to library and function authors making the intention plain rather than the language providing a specific construct for doing so. I find this acceptable, but others may not.
Yes, I realize this. I think the overhead of creating a closure would be acceptable provided it only affected this explicitly specified case and not function calls in general. I've long ago accepted that expressive power and efficiency are often mutually exclusive (which I why I moved from C to Python many years ago). Cliff

Cliff Wells wrote:
The problem is telling which cases are affected. If you can't tell until run time, then code has to be generated to handle either possible case for all function arguments, which seems like an effort out of proportion to the problem being solved. And there will still be some runtime overhead in selecting between the two cases. The only way I could see this working is if deferred arguments are marked at *both* the calling and called sites. Then the compiler knows what code to generate, and the called function can check that it has been passed the right kind of argument. -- Greg

On Fri, 2008-09-19 at 13:14 +1200, Greg Ewing wrote:
You should be able to tell at compile-time which functions are affected (assuming we used a signature indicator. If a function has *any* deferred arguments, it is a special type <function-with-deferred-arguments> rather than plain <function>.
And there will still be some runtime overhead in selecting between the two cases.
It depends on how selection is done already (say for functions vs methods). If it's implemented via a type-selected vector then having a separate type <function-with-deferred-arguments> then the overhead would be nil. If it requires explicit testing, then there would be at least a small overhead.
Actually, that might just be acceptable (if not my preferred form). It meets my desire to enforce lazy evaluation although it retains the ugly lambda in the caller. Cliff

I fail to see any advantage to introducing new syntax and semantics that are likely to confuse people when the capability to do this is already there in a simple and easy to use way. As I said earlier, mark it at the calling site with "lambda:" and at the point where it should be evaluated with "()". And no reasons have been offered for why this should is *so important* that it should be promoted beyond what's already there. Start from the problem, not from the "solution." I don't want to see python become rattlesnake -- where you have to watch carefully every place you step. --- Bruce

On Fri, Sep 19, 2008 at 3:08 PM, Cliff Wells <cliff@develix.com> wrote:
But once you accept that deferred arguments must be marked at the call site, you no longer need any additions to Python. The approach Bruce Leban suggested upthread is sufficient -- wrap your deferred arguments in a lambda, and call to expand them on the function side. This whole discussion, with the non-starter proposal of resurrecting backticks to solve a non-problem, reminds me of an eloquent post earlier this year on the Lua mailing list from Fabien Fleutot: http://lua-users.org/lists/lua-l/2008-02/msg00247.html A part of it applies here almost verbatim. With two changes to reflect Python instead: What drives people so crazy about it? It's not the couple of extra keystrokes, nor the screen real estate eaten by [lambda:a+b]; it's the fact that the current syntax carries a message they strongly disagree with, which says: "anonymous functions are not a lightweight feature to be used pervasively, passing them as function parameters is not the [Python] Way, that's why it looks syntactically odd. If you use them, it ought to be because you do something exceptional, so your code should have a somewhat exceptional appearance". They feel wrong when they use it, and they think they ought to feel good. This thread seems to be about technical solution to the "problem" that you can't really define functions to act as new control structures in Python. But while this may be a design choice you don't agree with, it's not an accident, and it's not a misfeature! Of course you could design a Python-like language that allowed users to create their own control structures. But this would be so different to not be at all Pythonic anymore! The fact that your new language would be a superset of Python is beside the point. Now every large project would have its own control structures defined. Readers of the resulting code will have to spend more time figuring out if the things that look like function calls actually are; reasoning about side effects just got a lot harder. This technical discussion is misguided. There isn't a problem here to fix to begin with. Greg F

On Fri, 2008-09-19 at 16:06 -0400, Greg Falcon wrote:
This is a pretty compelling argument. The reason this concept works so well in Io is because it's consistent (and is therefore unsurprising) whereas in Python it doesn't fit in the overall scheme of things (it's unexpected). I guess at the end of the day I'm just left with no longer liking the "Python Way". Fair enough. Cliff

Cliff Wells wrote:
Compiling the function is no problem. Compiling and running the function call is. The class of the object resulting from evaluating an expression is generally not known until runtime. For callable expressions, the only constant expression is a lambda expression.
Python has several callable classes. 'Function' and 'method' (3.0 name) are just two of them. When the compiler sees 'e1(e2)', it assume that the programmer intends e1 to evaluate to a callable and e2 to an argument thereof. So it compiles code to evaluate e1, evaluate e2, and then call e1 with arg e2. The 'selection', such as it is, takes place in the evaluation of the callable expression, by virtue of the class of the resulting object. But the interpreter pays no attention to that, but simply tries to call the presumed callable with the evaluated argument expression by looking for the callable's __call__ method. Which is to say, there is no selection as you seem to mean the term. It is a simple and elegant design, but without major change, not very amenable to your desire for runtime selection of arg processing and passing method.
There is neither a vector nor explicit testing. The 'callable' is called, and if it is not callable, the interpreter reports back TypeError: 'whatever' object is not callable Terry Jan Reedy

Cliff Wells wrote:
You should be able to tell at compile-time which functions are affected (assuming we used a signature indicator.
I'm talking about telling at the calling site whether an argument needs to be passed evaluated or unevaluated. You can' tell that at compile time without a marker at the calling site as well.
It would really be better to have a new syntax for the calling-site marker, and some kind of flag passed along with the argument behind the scenes. If you just rely on looking at whether the passed argument is a function, you can't tell the difference between a function representing an unevaluated argument and an evaluated argument whose value just happens to be a function. -- Greg

Cliff Wells wrote:
There's a fundamental problem which is fatal to this idea and others like it (which resulted in the death of many of my own suggestions over the years). The problem is that Python can't support "programmable syntax". Think about the mechanics of import. Unlike C++ or Java, the "import" statement doesn't actually do anything until the program is actually run. The compiler never sees the contents of the imported module. This means that any syntax-changing directives would have to be manually copied into each and every Python source file that used them. (In a sense, this is exactly how import future works.) There's no means to have a standard library of syntax-changing directives. -- Talin

I wonder, why don't you use haskell? Everything is an expression in haskell and everything is lazy. You should be happy with haskell. And there are even haskell compilers, so it's fast, too. ;) -panzi

On Wed, Sep 17, 2008, Cliff Wells wrote:
Approach to what? Please keep in mind that this list is archived; using a sensible Subject: line goes a long way toward making that archive useful and usable. (Plus it makes the list more usable in the present because not everyone who reads this list reads every post on it.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach

Cliff Wells wrote:
This is called "pass by name": <http://www.cs.sfu.ca/~cameron/Teaching/383/PassByName.html> Syntactically you need some parameter annotation that says you want pass-by-name semantics, then add a "thunk" layer when those parameters are used in the code. Note that in languages that support pass-by-name, I happen to be familiar with ALGOL, it operates on both the left and right hand side of assignment statements. Perhaps something like this: >>> def foo(arg : __byname__): ... print arg ... arg = (arg * 3) + 1 ... print arg ... >>> x = 2 >>> foo(x) 2 7 >>> print x 7
I'm with you, I think there are quite effective uses for this functionality, but nothing that will combat the wave of "why this is fundamentally wrong" or "there's a better way to do it" arguments. Joel

On Wed, Sep 17, 2008 at 3:41 PM, Cliff Wells <cliff@develix.com> wrote:
I'd rather use the time machine. You can already do this quite easily. def foo (arg): return arg() # evaluation happens here foo(lambda: x) or def IF(c,t,f=lambda:None): if c: return t() else: return f() IF(hasattr(spam, 'eggs'), lambda: spam.eggs, lambda: 'not found') The idea of something automagically changing from an unevaluated to an evaluated value strikes me as a very bad one so requiring explicit evaluation here is a *good* thing. Aside from that, why should arg be evaluated on return as opposed to being passed along unevaluated until its value is really needed? How would I check to see if an arg was provided? I certainly can't use a default value of None and then check for None since the check would cause it to be evaluated. Maybe we have a special operator that allows us to look at the unevaluated value. What's the context of the evaluation? Can it access variables in the place where it's evaluated? Maybe there's a special way to do that too. Let's not go down that path. --- Bruce

On Wed, 2008-09-17 at 16:18 -0700, Bruce Leban wrote:
Except I'd prefer not having lambda sprinkled throughout the calling code (or worse, forgotten).
Didn't suggest that. That was merely an example. What I'm suggesting is that the *first* time it's used, evaluate then (my example used return for simplicity).
I don't consider that a problem. You either care about the value of an argument or you don't. If you care then you will eventually evaluate it anyway. If you don't, then don't evaluate it, ever.
Agreed we shouldn't go down that path, but don't agree that it's needed. Cliff

On Wed, Sep 17, 2008 at 5:27 PM, Cliff Wells <cliff@develix.com> wrote:
Back when you were talking about statements -> expressions, I was going to point out deferred function calls, but I also ran into the "lambda is ugly" problem. If it weren't for the fact that backticks ` are difficult to distinguish from regular single quotes ', never mind being a PITA to type on some keyboards, I would *almost* suggest that backticks should be used for expressing "a function without arguments, aka a deferred execution expression". But then I'd have to give myself a -1, as it violates TOOWTDI. Then again, the argument could be made that lambdas are just a long spelling of a def without a name, and that ... add = def (arg1, arg2): arg1+arg2 would be better. At that point, we come to def:'deferred' . And I think that's ever worse than backticks. I don't know of a spelling of deferred execution blocks that I would find reasonable :/ . - Josiah

Josiah Carlson wrote:
I don't know of a spelling of deferred execution blocks that I would find reasonable :/ .
His request is worse than that. He didn't ask for deferred execution, he asked for deferred values. In python Right Now, you can already do deferred execution blocks (forgetting the "I don't like seeing lambdas!" argument). Rather, he wants to be able to have arbitrary expressions be treated as lazy, not at the call-site either but determined by the receiver: """ I'd prefer to write def foo ( lambda: arg ): return arg # evaluation happens here foo ( x ) """ I'm sorry but this will never, ever, ever happen in Python. That is such a *huge* shift in semantics that you should go use a different language. In such languages, it is *impossible* to know when the expression will actually be evaluated, and while this is fine in a purely functional environment, this is not easy to deal with in imperative programming. If you write "foo(bar())", then you have no way of knowing whether "bar()" is ever run. And in a world that depends on side-effects, that forces you to force evaluations of expressions. You would be forced to write "x = bar(); foo(x)" (assuming assignment isn't lazy), which is less clear than having to write "foo(lambda: bar())" in the opposite case. Explicit is better than implicit. -Scott -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu

On Wed, Sep 17, 2008 at 8:05 PM, Scott Dial <scott+python-ideas@scottdial.com> wrote:
What I was thinking (completely unreasonable, I would be -1 on this) is that the following two lines would be equivalent... x = lambda: foo.bar x = `foo.bar` Sadly, that doesn't really gain you anything - though it would be convenient in the case of an If function (or similar): def If(condition, truev, falsev=`None`): if condition: return truev() return falsev() result = If(x < 5, `foo.smallx(x)`, `foo.largex(x)`) Though, because it's still a function call, it's still slow. And even worse, uses a previously decided-upon ugly pair of characters. While I understand that Cliff wanted auto-execution, I think that requiring a call isn't out of the question (what do Ruby thunks require?) - Josiah

On Wed, 2008-09-17 at 16:18 -0700, Bruce Leban wrote:
To be clear, the problem is that this leads to brittle code (I want to *enforce* lazy evaluation in a certain context): IF(a<100, b**a, 0) Oops, forgot lambda and nothing crashes but we now constantly evaluate unbounded values.
Sorry, missed answering this: the context would be unchanged. It would be evaluated as if it were evaluated at the time of the function call. This may be a serious implementation problem, but I'll let the experts answer that. Regards, Cliff

Cliff Wells wrote:
Unless some clever implementation trick could be found, this would have extremely serious performance implications in CPython. Every expression used as a function argument would have to be compiled as a separate function. A closure would need to be created every time it was used, and a Python call executed every time it was evaluated, both fairly expensive operations. Any variables it uses from the surrounding scope would need to be allocated in cells, meaning more memory allocations and access overhead. I could imagine this making the execution of *all* Python code several times slower or worse. -- Greg

Cliff Wells wrote:
You can accomplish this to some degree by using lambda,
You can do it completely as far as I know.
but I'd much prefer something indicated in the function signature than in the calling code.
Given Python's nature as a compiled dynamic language, indications are really needed in both caller and callee, as at present: pass a function and call it. Without an explicit indication at the call site, the compiler would have to compile both direct evaluation and function object creation and the code to select between them at runtime according to an attribute of the function resulting from evaluation of the presumed function expression. This would slow *all* function calls, which are slow enough already. Also, one could not tell what objects get passed to a function without knowing the signature in detail, making code harder to read. There is already a way to be flexible without magical arg object switching and with only exceptional need for del/lambda. def iff(cond, if_result, else_result): if cond: if hasattr(if_result, '__call__'): return if_result() else: return if_result else: if hasattr(else_result, '__call__'): return else_result() else: return else_result for i in [0,1,2]: print("I gave you %s gold piece%s." % (i,iff(i==1, '', 's'))) #3.0 #prints (imagine a typical RPG game) I gave you 0 gold pieces. I gave you 1 gold piece. I gave you 2 gold pieces. One only needs the wrapper if the object to be returned is a function example -- iff(cond, lambda: math.sin, lambda: math.cos)(x) or if the expression needs to be guarded by the condition because it would otherwise be invalid (iff(x, lambda x=x: sin(x)/x, 1) -- this might work without the default arg, but might have scope problems) or if it would take a long time. But many conditional values are constant or quick to evaluate and the short-circuiting is not needed. Many of the exceptions might better be defined in functions anyway. def sin_or_cos(cond,x): if cond: return math.sin(x) else: return math.cos(x) def sin_over_x(x): if x: return math.sin(x)/x else: return 0 Terry J. Reedy

On Wed, 2008-09-17 at 23:54 -0400, Terry Reedy wrote:
Yes, I had this suspicion, but I figured I'd wait for an authority to make it plain. However, consider this: foo ( a + b ) # a + b is compiled but not currently evaluated Now, clearly Python would need to know at runtime if foo() accepts a lazy argument prior to calling it, however I'm reminded that Python 3.0 is adding optional static type checking (I believe) which seems to bring the same issue to the table (and possibly at the same expense you mention). Would the mechanism not be more or less the same? Check signature and then apply arguments? Cliff

On Wed, 2008-09-17 at 23:54 -0400, Terry Reedy wrote:
There's one other approach: have two types of functions: <function> and <function-with-lazy-args>. Whenever Python encounters a declaration of a function with a lazy argument it creates the latter. This would allow Python to know at runtime how to handle it. The difference could be transparent to user code (although it might be unappealing to disguise such a difference). Cliff

On Thu, Sep 18, 2008 at 7:01 AM, Cliff Wells <cliff@develix.com> wrote:
Why don't you just use strings instead of actual arguments and then just evaluate them if you need them? Someone has already suggested that... As I briefly understand your idea of making Python a more functional language (I like FP myself), I don't think Python is ever going to be one. If you want some FP, just use another language (lazy args will not be implemented ever for sure) Or you can fork Python :) Cheers, Peter

Cliff Wells wrote:
This is what C and languages sort of have with functions and text macros. Without a name convention (such as CPython uses in its codebase, with macros ALL_CAPS, I believe), one cannot tell, for instance, whether f(x) will or will not result in the passing of 'x' or eval(x). It is what some Lisps used to have -- functions and special functions. Users had to memorized which args of which functions were normal and lazy. There must have been some reason why that was superceded. But to repeat: with Python's dynamic name binding, the compiler cannot in general tell whether the 'f' in 'f(x)' will be bound at runtime to a regular or special function, so as I said above, it would have to code for both possibilities.
This was already assumed in my original 'select between them at runtime according to an attribute of the function'.
Unless the argument expression has visible side-effects when evaluated. I think there are also namespace issues that you have missed. Consider def f(`a,`b): # ` indicates lazy evaulation, after Lisp's 'quote ..... blaf + f(a+b,c) -3 When f evaluates a bound to 'a+b', it has to *not* resolve 'b' to f's 'b' but to the caller's 'b'. As someone else said, the unevaluated code for 'a+b' is not enough. A full enclosing function is needed. tjr

On Thu, 2008-09-18 at 16:10 -0400, Terry Reedy wrote:
Cliff Wells wrote:
Sorry, I must have missed that in reading your reply, but yes both would work. Which of them would be preferable is an implementation detail outside my scope of knowledge.
I think this is the stickiest point (from a philosophical perspective anyway). It's true an explicit side-effect might not happen. However, we already have the inverse of this problem (simple expressions disguising side-effects). Consider: a + b Given that a and be might not be numeric types, what does the above mean? Someone mentioned earlier (sorry, lost the reference) that they found the idea of having to refer to a function signature to unappealing, but magic methods already present a similar problem (disguising potential side-effects). We depend on sane library authors to not abuse this feature. In any case, whenever encountering such a construct we are forced to either assume that the class author did the right thing and the operation makes sense or we refer to the documentation/source to decipher the meaning. I think what I'm suggesting isn't so dissimilar. I will admit that the mere *presence* of a class implies that such side-effects might be present whereas functions only appear ordinary, so there is a tiny bit of fair warning. Still, when encountering something like iff ( cond, iftrue, iffalse ) in a library, I think what I'm suggesting is the least surprising result. It comes down to library and function authors making the intention plain rather than the language providing a specific construct for doing so. I find this acceptable, but others may not.
Yes, I realize this. I think the overhead of creating a closure would be acceptable provided it only affected this explicitly specified case and not function calls in general. I've long ago accepted that expressive power and efficiency are often mutually exclusive (which I why I moved from C to Python many years ago). Cliff

Cliff Wells wrote:
The problem is telling which cases are affected. If you can't tell until run time, then code has to be generated to handle either possible case for all function arguments, which seems like an effort out of proportion to the problem being solved. And there will still be some runtime overhead in selecting between the two cases. The only way I could see this working is if deferred arguments are marked at *both* the calling and called sites. Then the compiler knows what code to generate, and the called function can check that it has been passed the right kind of argument. -- Greg

On Fri, 2008-09-19 at 13:14 +1200, Greg Ewing wrote:
You should be able to tell at compile-time which functions are affected (assuming we used a signature indicator. If a function has *any* deferred arguments, it is a special type <function-with-deferred-arguments> rather than plain <function>.
And there will still be some runtime overhead in selecting between the two cases.
It depends on how selection is done already (say for functions vs methods). If it's implemented via a type-selected vector then having a separate type <function-with-deferred-arguments> then the overhead would be nil. If it requires explicit testing, then there would be at least a small overhead.
Actually, that might just be acceptable (if not my preferred form). It meets my desire to enforce lazy evaluation although it retains the ugly lambda in the caller. Cliff

I fail to see any advantage to introducing new syntax and semantics that are likely to confuse people when the capability to do this is already there in a simple and easy to use way. As I said earlier, mark it at the calling site with "lambda:" and at the point where it should be evaluated with "()". And no reasons have been offered for why this should is *so important* that it should be promoted beyond what's already there. Start from the problem, not from the "solution." I don't want to see python become rattlesnake -- where you have to watch carefully every place you step. --- Bruce

On Fri, Sep 19, 2008 at 3:08 PM, Cliff Wells <cliff@develix.com> wrote:
But once you accept that deferred arguments must be marked at the call site, you no longer need any additions to Python. The approach Bruce Leban suggested upthread is sufficient -- wrap your deferred arguments in a lambda, and call to expand them on the function side. This whole discussion, with the non-starter proposal of resurrecting backticks to solve a non-problem, reminds me of an eloquent post earlier this year on the Lua mailing list from Fabien Fleutot: http://lua-users.org/lists/lua-l/2008-02/msg00247.html A part of it applies here almost verbatim. With two changes to reflect Python instead: What drives people so crazy about it? It's not the couple of extra keystrokes, nor the screen real estate eaten by [lambda:a+b]; it's the fact that the current syntax carries a message they strongly disagree with, which says: "anonymous functions are not a lightweight feature to be used pervasively, passing them as function parameters is not the [Python] Way, that's why it looks syntactically odd. If you use them, it ought to be because you do something exceptional, so your code should have a somewhat exceptional appearance". They feel wrong when they use it, and they think they ought to feel good. This thread seems to be about technical solution to the "problem" that you can't really define functions to act as new control structures in Python. But while this may be a design choice you don't agree with, it's not an accident, and it's not a misfeature! Of course you could design a Python-like language that allowed users to create their own control structures. But this would be so different to not be at all Pythonic anymore! The fact that your new language would be a superset of Python is beside the point. Now every large project would have its own control structures defined. Readers of the resulting code will have to spend more time figuring out if the things that look like function calls actually are; reasoning about side effects just got a lot harder. This technical discussion is misguided. There isn't a problem here to fix to begin with. Greg F

On Fri, 2008-09-19 at 16:06 -0400, Greg Falcon wrote:
This is a pretty compelling argument. The reason this concept works so well in Io is because it's consistent (and is therefore unsurprising) whereas in Python it doesn't fit in the overall scheme of things (it's unexpected). I guess at the end of the day I'm just left with no longer liking the "Python Way". Fair enough. Cliff

Cliff Wells wrote:
Compiling the function is no problem. Compiling and running the function call is. The class of the object resulting from evaluating an expression is generally not known until runtime. For callable expressions, the only constant expression is a lambda expression.
Python has several callable classes. 'Function' and 'method' (3.0 name) are just two of them. When the compiler sees 'e1(e2)', it assume that the programmer intends e1 to evaluate to a callable and e2 to an argument thereof. So it compiles code to evaluate e1, evaluate e2, and then call e1 with arg e2. The 'selection', such as it is, takes place in the evaluation of the callable expression, by virtue of the class of the resulting object. But the interpreter pays no attention to that, but simply tries to call the presumed callable with the evaluated argument expression by looking for the callable's __call__ method. Which is to say, there is no selection as you seem to mean the term. It is a simple and elegant design, but without major change, not very amenable to your desire for runtime selection of arg processing and passing method.
There is neither a vector nor explicit testing. The 'callable' is called, and if it is not callable, the interpreter reports back TypeError: 'whatever' object is not callable Terry Jan Reedy

Cliff Wells wrote:
You should be able to tell at compile-time which functions are affected (assuming we used a signature indicator.
I'm talking about telling at the calling site whether an argument needs to be passed evaluated or unevaluated. You can' tell that at compile time without a marker at the calling site as well.
It would really be better to have a new syntax for the calling-site marker, and some kind of flag passed along with the argument behind the scenes. If you just rely on looking at whether the passed argument is a function, you can't tell the difference between a function representing an unevaluated argument and an evaluated argument whose value just happens to be a function. -- Greg

Cliff Wells wrote:
There's a fundamental problem which is fatal to this idea and others like it (which resulted in the death of many of my own suggestions over the years). The problem is that Python can't support "programmable syntax". Think about the mechanics of import. Unlike C++ or Java, the "import" statement doesn't actually do anything until the program is actually run. The compiler never sees the contents of the imported module. This means that any syntax-changing directives would have to be manually copied into each and every Python source file that used them. (In a sense, this is exactly how import future works.) There's no means to have a standard library of syntax-changing directives. -- Talin

I wonder, why don't you use haskell? Everything is an expression in haskell and everything is lazy. You should be happy with haskell. And there are even haskell compilers, so it's fast, too. ;) -panzi

On Wed, Sep 17, 2008, Cliff Wells wrote:
Approach to what? Please keep in mind that this list is archived; using a sensible Subject: line goes a long way toward making that archive useful and usable. (Plus it makes the list more usable in the present because not everyone who reads this list reads every post on it.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach

Cliff Wells wrote:
This is called "pass by name": <http://www.cs.sfu.ca/~cameron/Teaching/383/PassByName.html> Syntactically you need some parameter annotation that says you want pass-by-name semantics, then add a "thunk" layer when those parameters are used in the code. Note that in languages that support pass-by-name, I happen to be familiar with ALGOL, it operates on both the left and right hand side of assignment statements. Perhaps something like this: >>> def foo(arg : __byname__): ... print arg ... arg = (arg * 3) + 1 ... print arg ... >>> x = 2 >>> foo(x) 2 7 >>> print x 7
I'm with you, I think there are quite effective uses for this functionality, but nothing that will combat the wave of "why this is fundamentally wrong" or "there's a better way to do it" arguments. Joel
participants (12)
-
Aahz
-
Bruce Leban
-
Cliff Wells
-
Greg Ewing
-
Greg Falcon
-
Joel Bender
-
Josiah Carlson
-
Mathias Panzenböck
-
Piotr Wysocki
-
Scott Dial
-
Talin
-
Terry Reedy