At 11:55 AM 04/19/2005 -0700, Guido van Rossum wrote:

I'd recommend this:

tri = self.subcalculation("The quick brown fox jumps over the lazy dog") self.disentangle(0x40, tri, self.indent+1)

IMO this is clearer, and even shorter!

What was your opinion on "where" as a lambda replacement? i.e. foo = bar(callback1, callback2) where: def callback1(x): print "hello, " def callback2(x): print "world!" I suspect that you like the define-first approach because of your tendency to ask questions first and read later. That is, you want to know what callback1 and callback2 are before you see them passed to something. However, other people seem to like to have the context first, then fill in the details of each callback later. Interestingly, this syntax also works to do decoration, though it's not a syntax that was ever proposed for that. e.g.: foo = classmethod(foo) where: def foo(cls,x,y,z): # etc. foo = property(get_foo,set_foo) where: def get_foo(self): # ... def set_foo(self): # ... I don't mind @decorators, of course, but maybe they wouldn't be needed here.

If we apply this to the anonymous block problem, we may end up finding lambda the ultimate compromise -- like a gentleman in the back of my talk last week at baypiggies observed (unfortunately I don't know his name).

-- --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com

What was your opinion on "where" as a lambda replacement? i.e.

foo = bar(callback1, callback2) where: def callback1(x): print "hello, " def callback2(x): print "world!"

I don't recall seeing this proposed, but I might have -- I thought of pretty much exactly this syntax in the shower a few days ago. Unfortunately it doesn't solve the lock-release use case that is more pressing in my mind. Also, if you want top-down programming (which is a fine coding style!), we already have several ways to do that.

I suspect that you like the define-first approach because of your tendency to ask questions first and read later. That is, you want to know what callback1 and callback2 are before you see them passed to something. However, other people seem to like to have the context first, then fill in the details of each callback later.

I think it all depends, not so much on the personality of the reader, but on the specifics of the program. When callback1 and callback2 are large chunks of code, we probably all agree that it's better to have them out of the way, either way up or way down -- purely because of their size they deserve to be abstracted away when we're reading on how they are being used. A more interesting use case may be when callback1 and callback2 are very *small* amounts of code, since that's the main use case for lambda; there knowing what callback1 and callback2 stand for is probably important. I have to say that as long as it's only a few lines away I don't care much whether the detail is above or below its application, since it will all fit on a single screen and I can look at it all together. So then the 'where' syntax isn't particularly attractive because it doesn't solve a problem I'm experiencing.

Interestingly, this syntax also works to do decoration, though it's not a syntax that was ever proposed for that. e.g.:

foo = classmethod(foo) where: def foo(cls,x,y,z): # etc.

This requires you to write foo three times, which defeats at least half of the purpose of decorators.

foo = property(get_foo,set_foo) where: def get_foo(self): # ... def set_foo(self): # ...

I don't mind @decorators, of course, but maybe they wouldn't be needed here.

As I said before, I'm not sure why keeping get_foo etc. out of the class namespace is such a big deal. In fact, I like having them there (sometimes they can even be handy, e.g. you might be able to pass the unbound get_foo method as a sort key). -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:

As I said before, I'm not sure why keeping get_foo etc. out of the class namespace is such a big deal. In fact, I like having them there (sometimes they can even be handy, e.g. you might be able to pass the unbound get_foo method as a sort key).

Not to mention that it's possible to override get_foo in subclasses if done right ... Two approaches are here: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/408713 Tim Delaney

I apologize for sparking such debate on this list instead of on c.l.py. By the way, the only reason I brought this up was as a replacement for lambdas in Py3K. Guido, in response to your much earlier comment about supporting "{}" for normal defs as a matter of consistency within my proposal, yes, I agree. Just like ";", you should rarely use them. Best Regards, -jj On 4/19/05, Phillip J. Eby <pje@telecommunity.com> wrote:

At 01:00 PM 04/19/2005 -0700, Guido van Rossum wrote:

...

...

Interestingly, this syntax also works to do decoration, though it's not a syntax that was ever proposed for that. e.g.:

foo = classmethod(foo) where: def foo(cls,x,y,z): # etc.

This requires you to write foo three times, which defeats at least half of the purpose of decorators.

Well, you could do 'foo = classmethod(x) where: def x(...)', but that *is* kind of kludgy. I'm just suggesting that if 'where:' had existed before decorators, people might have griped about the three-time typing or kludged around it, but there wouldn't likely have been strong support for creating a syntax "just" for decorators.

Indeed, if somebody had proposed this syntax during the decorator debates I would have supported it, but of course Bob Ippolito (whose PyObjC use cases involve really long function names) might have disagreed.

...

...

foo = property(get_foo,set_foo) where: def get_foo(self): # ... def set_foo(self): # ...

I don't mind @decorators, of course, but maybe they wouldn't be needed here.

As I said before, I'm not sure why keeping get_foo etc. out of the class namespace is such a big deal.

That's a relatively minor thing, compared to being able to logically group them with the property, which I think enhances readability, even more than the sometimes-proposed '@property.getter' and '@property.setter' decorators.

Anyway, just to be clear, I don't personally think 'where:' is needed in Python 2.x; lambda and decorators suffice for all but the most Twisted use cases. ;) I was just viewing it as a potential alternative to lambda in Py3K.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/jjinux%40gmail.com

-- I have decided to switch to Gmail, but messages to my Yahoo account will still get through.

...

IMO this is clearer, and even shorter! But it clutters the namespace with objects you don't need.

Why do people care about cluttering namespaces so much? I thought thats' what namespaces were for -- to put stuff you want to remember for a bit. A function's local namespace in particular seems a perfectly fine place for temporaries. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:

...

...

IMO this is clearer, and even shorter!

But it clutters the namespace with objects you don't need.

Why do people care about cluttering namespaces so much? I thought thats' what namespaces were for -- to put stuff you want to remember for a bit. A function's local namespace in particular seems a perfectly fine place for temporaries.

Indeed. The way people bang on about "cluttering namespaces" you'd be forgiven for thinking that they are like attics, permanently attached to the house and liable to become cluttered over years. Most function namespaces are in fact extremely short-lived, and there is little point worrying about clutter as long as there's no chance of confusion. regards Steve -- Steve Holden +1 703 861 4237 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/

Glyph Lefkowitz wrote:

Despite being guilty of propagating this style for years myself, I have to disagree. Consider the following network-conversation using Twisted style (which, I might add, would be generalizable to other Twisted-like systems if they existed ;-)):

def strawman(self): def sayGoodbye(mingleResult): def goAway(goodbyeResult): self.loseConnection() self.send("goodbye").addCallback(goAway) def mingle(helloResult): self.send("nice weather we're having").addCallback(sayGoodbye) self.send("hello").addCallback(mingle)

def iterman(self): yield "hello" yield "nice weather we're having" yield "goodbye" </F>

Bob Ippolito wrote:

...

...

def strawman(self): def sayGoodbye(mingleResult): def goAway(goodbyeResult): self.loseConnection() self.send("goodbye").addCallback(goAway) def mingle(helloResult): self.send("nice weather we're having").addCallback(sayGoodbye) self.send("hello").addCallback(mingle)

def iterman(self): yield "hello" yield "nice weather we're having" yield "goodbye"

Which, more or less works, for a literal translation of the straw-man above. However, you're missing the point. These deferred operations actually return results. Generators offer no sane way to pass results back in.

that's why you need a context object (=self, in this case). def iterman(self): yield "hello" print self.data yield "nice weather we're having" print self.data yield "goodbye" also see: http://effbot.org/zone/asyncore-generators.htm

If they did, then this use case could be mostly served by generators.

exactly. </F>

See the thread "pre-PEP: Suite-Based Keywords" (shamless plug) (an earlier, similar proposal is here: http://groups.google.co.uk/groups?selm=mailman.403.1105274631.22381.python-l... %40python.org ).

In short, if doFoo is defined like:

def doFoo(func1, func2): pass

You would be able to call it like:

doFoo(**): def func1(a, b): return a + b def func2(c, d): return c + d

That is, a suite can be used to define keyword arguments.

I'm still not sure how this is particularly solving a pressing problem that isn't solved by putting the function definitions in front of the call. I saw the first version of the proto-PEP and didn't think that the motivating example (keeping the getx/setx methods passed to a property definition out of the class namespace) was all that valuable. Two more issues: (1) It seems that *every* name introduced in the block automatically becomes a keyword argument. This looks like a problem, since you could easily need temporary variables there. (I don't see that a problem with class bodies because the typical use there is only method and property definitions and the occasional instance variable default.) (2) This seems to be attaching a block to a specific function call but there are more general cases: e.g. you might want to assign the return value of doFoo() to a variable, or you might want to pass it as an argument to another call. *If* we're going to create syntax for anonymous blocks, I think the primary use case ought to be cleanup operations to replace try/finally blocks for locking and similar things. I'd love to have syntactical support so I can write blahblah(myLock): code code code instead of myLock.acquire() try: code code code finally: myLock.release() -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:

...

See the thread "pre-PEP: Suite-Based Keywords" (shamless plug) (an earlier, similar proposal is here: http://groups.google.co.uk/groups?selm=mailman.403.1105274631.22381.python-l... %40python.org ).

In short, if doFoo is defined like:

def doFoo(func1, func2): pass

You would be able to call it like:

doFoo(**): def func1(a, b): return a + b def func2(c, d): return c + d

That is, a suite can be used to define keyword arguments.

I'm still not sure how this is particularly solving a pressing problem that isn't solved by putting the function definitions in front of the call. I saw the first version of the proto-PEP and didn't think that the motivating example (keeping the getx/setx methods passed to a property definition out of the class namespace) was all that valuable.

OK. I think most people (myself included) who would prefer to define properties (and event handlers, etc.) in this way are motivated by the perception that the current method is just ugly. I don't know that it solves any pressing problems.

Two more issues:

(1) It seems that *every* name introduced in the block automatically becomes a keyword argument. This looks like a problem, since you could easily need temporary variables there. (I don't see that a problem with class bodies because the typical use there is only method and property definitions and the occasional instance variable default.)

Combining the suite-based keywords proposal with the earlier, 'where' proposal (linked in my above post), you would be able to name variables individually in the case that temporary variables are needed: f(x=x): x = [i**2 for i in [1,2,3]]

(2) This seems to be attaching a block to a specific function call but there are more general cases: e.g. you might want to assign the return value of doFoo() to a variable, or you might want to pass it as an argument to another call.

The 'where' proposal also doesn't have this problem. Any expression is allowed.

*If* we're going to create syntax for anonymous blocks, I think the primary use case ought to be cleanup operations to replace try/finally blocks for locking and similar things. I'd love to have syntactical support so I can write

blahblah(myLock): code code code

instead of

myLock.acquire() try: code code code finally: myLock.release()

Well, that was my other proposal, "pre-PEP: Simple Thunks" (there is also an implementation). It didn't seem to go over all that well. I am going to try to rewrite it and give more motivation and explanation (and maybe use 'with' and 'from' instead of 'do' and 'in' as keywords). -Brian

@acquire(myLock): code code code

It would certainly solve the problem of which keyword to use! :-) And I think the syntax isn't even ambiguous -- the trailing colon distinguishes this from the function decorator syntax. I guess it would morph '@xxx' into "user-defined-keyword". How would acquire be defined? I guess it could be this, returning a function that takes a callable as an argument just like other decorators: def acquire(aLock): def acquirer(block): aLock.acquire() try: block() finally: aLock.release() return acquirer and the substitution of @EXPR: CODE would become something like def __block(): CODE EXPR(__block) I'm not yet sure whether to love or hate it. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Why not have the block automatically be inserted into acquire's argument list? It would probably get annoying to have to define inner functions like that every time one simply wants to use arguments.

But the number of *uses* would be much larger than the number of "block decorators" you'd be coding. If you find yourself writing new block decorators all the time that's probably a sign you're too much in love with the feature. :-)

For example:

def acquire(block, aLock): aLock.acquire() try: block() finally: aLock.release()

@acquire(myLock): code code code

Of course, augmenting the argument list in that way would be different than the behavior of decorators as they are now.

I don't like implicit modifications of argument lists other than by method calls. It's okay for method calls because in the x.foo(a) <==> foo(x, a) equivalence, x is really close to the beginning of the argument list. And your proposal would preclude parameterless block decorators (or turn them into an ugly special case), which I think might be quite useful: @forever: infinite loop body @ignore: not executed at all @require: assertions go here and so on. (In essence, we're inventing the opposite of "barewords" in Perl here, right?) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On 4/19/05, Brian Sabbey <sabbey@u.washington.edu> wrote:

Guido van Rossum wrote:

...

...

@acquire(myLock): code code code

It would certainly solve the problem of which keyword to use! :-) And I think the syntax isn't even ambiguous -- the trailing colon distinguishes this from the function decorator syntax. I guess it would morph '@xxx' into "user-defined-keyword".

Hmm, this looks to me like a natural extension of decorators. Whether that is a good or a bad thing, I'm unable to decide :-) [I can think of a number of uses for it, PEP 310-style with-blocks being one, but I can't decide if "lots of potential uses" is too close to "lots of potential for abuse" :-)]

...

How would acquire be defined? I guess it could be this, returning a function that takes a callable as an argument just like other decorators:

def acquire(aLock): def acquirer(block): aLock.acquire() try: block() finally: aLock.release() return acquirer

It really has to be this, IMO, otherwise the parallel with decorators becomes confusing, rather than helpful.

...

and the substitution of

@EXPR: CODE

would become something like

def __block(): CODE EXPR(__block)

The question of whether assignments within CODE are executed within a new namespace, as this implies, or in the surrounding namespace, remains open. I can see both as reasonable (new namespace = easier to describe/understand, more in line with decorators, probably far easier to implement; surrounding namespace = probably more useful/practical...)

Why not have the block automatically be inserted into acquire's argument list? It would probably get annoying to have to define inner functions like that every time one simply wants to use arguments.

If this syntax is to be considered, in my view it *must* follow established decorator practice - and that includes the define-an-inner-function-and-return-it idiom.

Of course, augmenting the argument list in that way would be different than the behavior of decorators as they are now.

Exactly. Paul.

James Y Knight wrote:

If it was possible to assign to a variable to a variable bound outside your function, but still in your lexical scope, I think it would fix this issue. That's always something I've thought should be possible, anyways. I propose to make it possible via a declaration similar to 'global'.

E.g. (stupid example, but it demonstrates the syntax): def f(): count = 0 def addCount(): lexical count count += 1 assert count == 0 addCount() assert count == 1

It strikes me that with something like this lexical declaration, we could abuse decorators as per Carl Banks's recipe[1] to get the equivalent of thunks: def withfile(filename, mode='r'): def _(func): f = open(filename, mode) try: func(f) finally: f.close() return _ and used like: line = None @withfile("readme.txt") def print_readme(fileobj): lexical line for line in fileobj: print line print "last line:" line As the recipe notes, the main difference between print_readme and a real "code block" is that print_readme doesn't have access to the lexical scope. Something like James's suggestion would solve this problem. One advantage I see of this route (i.e. using defs + lexical scoping instead of new syntactic support) is that because we're using a normal function, the parameter list is not an issue -- arguments to the "thunk" are bound to names just as they are in any other function. The big disadvantage I see is that my normal expectations for decorators are wrong here -- after the decorator is applied print_readme is set to None, not a new callable object. Guess I'm still riding the fence. ;-) STeVe [1]http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/391199 -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy

On 4/21/05, Guido van Rossum <gvanrossum@gmail.com> wrote:

...

It strikes me that with something like this lexical declaration, we could abuse decorators as per Carl Banks's recipe[1] to get the equivalent of thunks:

"abuse" being the operative word.

Yup. I was just drawing the parallel between: @withfile("readme.txt") def thunk(fileobj): for line in fileobj: print line and @withfile("readme.txt"): # called by withfile as thunk(fileobj=<file object>) for line in fileobj: print line STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy

[Guido van Rossum]

@EXPR: CODE

would become something like

def __block(): CODE EXPR(__block)

I'm not yet sure whether to love or hate it. :-)

Is it preferable for CODE to execute in its own namespace (the above being a literal translation of the given code), or for it to execute in the originally defined namespace? Deferring to Greg Ewing for a moment [1]: They should be lexically scoped, not dynamically scoped. Wrapped blocks in an old namespace, I believe, is the way to go, especially for things like... @synchronize(fooLock): a = foo.method() I cannot come up with any code for which CODE executing in its own namespace makes sense. Can anyone else? <discussion on the overlap with PEP 310 removed for brevity> - Josiah [1] http://mail.python.org/pipermail/python-dev/2005-March/052239.html

RSMotD (random stupid musing of the day): so I wonder if the decorator syntax couldn't be extended for this kind of thing.

@acquire(myLock): code code code

Would it be useful for anything other than mutex-locking? And wouldn't it be better to make a function of the block wrapped in a block-decorator and then use a normal decorator? -- mvh Björn

Michael Walter <michael.walter@gmail.com> wrote:

On 4/19/05, BJörn Lindqvist <bjourne@gmail.com> wrote:

...

...

RSMotD (random stupid musing of the day): so I wonder if the decorator syntax couldn't be extended for this kind of thing.

@acquire(myLock): code code code

Would it be useful for anything other than mutex-locking? And wouldn't it be better to make a function of the block wrapped in a block-decorator and then use a normal decorator?

Yes. Check how blocks in Smalltalk and Ruby are used for starters.

See the previous two discussions on thunks here on python-dev, and notice how the only problem that seem bettered via blocks/thunks /in Python/ are those which are of the form... #setup try: block finally: #finalization ... and depending on the syntax, properties. I once asked "Any other use cases for one of the most powerful features of Ruby, in Python?" I have yet to hear any sort of reasonable response. Why am I getting no response to my question? Either it is because I am being ignored, or no one has taken the time to translate one of these 'killer features' from Smalltalk or Ruby, or perhaps such translations show that there is a better way in Python already. Now, don't get me wrong, I have more than a few examples of the try/finally block in my code, so I would personally find it useful, but just because this one pattern is made easier, doesn't mean that it should see syntax. - Josiah P.S. If I'm sounding like a broken record to you, don't be surprised. But until my one question is satisfactorally answered, I'll keep poking at its soft underbelly.

PS. a side effect of the for-in pattern is that I'm beginning to feel that Python might need a nice "switch" statement based on dictionary lookups, so I can replace multiple callbacks with a single loop body, without writing too many if/elif clauses.

That's funny. I keep wondering if "match" from the ML world would make sense in Python. I keep thinking it'd be a really nice thing to have. -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through.

Samuele Pedroni <pedronis@strakt.com> wrote:

...

def do(): print "setup" try: yield None finally: print "tear down"

doesn't quite work (if it did, all you would need is syntactic sugar for "for dummy in").

PEP325 is about that

PEP 288 can be used like that. - Josiah

"Fredrik Lundh" <fredrik@pythonware.com> wrote:

Josiah Carlson wrote:

...

See the previous two discussions on thunks here on python-dev, and notice how the only problem that seem bettered via blocks/thunks /in Python/ are those which are of the form...

#setup try: block finally: #finalization

... and depending on the syntax, properties. I once asked "Any other use cases for one of the most powerful features of Ruby, in Python?" I have yet to hear any sort of reasonable response.

Why am I getting no response to my question? Either it is because I am being ignored, or no one has taken the time to translate one of these 'killer features' from Smalltalk or Ruby, or perhaps such translations show that there is a better way in Python already.

for my purposes, I've found that the #1 callback killer in contemporary Python is for-in:s support for the iterator protocol:

...

and get shorter code that runs faster. (see cElementTree's iterparse for an excellent example. for typical use cases, it's nearly three times faster than pyexpat, which is the fastest callback-based XML parser we have)

It seems as though you are saying that because callbacks are so slow, that blocks are a non-starter for you because of how slow it would be to call them. I'm thinking that if people get correct code easier, that speed will not be as much of a concern (that's why I use Python already). With that said, both blocks and iterators makes /writing/ such things easier to understand, but neither really makes /reading/ much easier. Sure, it is far more terse, but that doesn't mean it is easier to read and understand what is going on. Which would people prefer? @a(l): code or l.acquire() try: code finally: l.release()

unfortunately,

def do(): print "setup" try: yield None finally: print "tear down"

doesn't quite work (if it did, all you would need is syntactic sugar for "for dummy in").

The use 'for dummy in...' would be sufficient to notify everyone. If 'dummy' is too long, there is always '_'. This kind of thing solves the common case of setup/finalization, albeit in a not-so-obvious-to-an-observer mechanism, which was recently loathed by a nontrivial number of python-dev posters (me being one). Looking at it again, a month or so later, I don't know. It does solve the problem, but it introduces a semantic where iteration is used for something that is not really iteration. Regardless, I believe that solving generator finalization (calling all enclosing finally blocks in the generator) is a worthwhile problem to solve. Whether that be by PEP 325, 288, 325+288, etc., that should be discussed. Whether people use it as a pseudo-block, or decide that blocks are further worthwhile, I suppose we could wait and see.

</F>

PS. a side effect of the for-in pattern is that I'm beginning to feel that Python might need a nice "switch" statement based on dictionary lookups, so I can replace multiple callbacks with a single loop body, without writing too many if/elif clauses.

If I remember correctly, Raymond was working on a peephole optimization that automatically translated if/elif/else clauses to a dictionary lookup when the objects were hashable and only the == operator was used. I've not heard anything about it in over a month, but then again, I've not finished the implementation of an alternate import semantic either. - Josiah

Fredrik Lundh wrote:

...

Regardless, I believe that solving generator finalization (calling all enclosing finally blocks in the generator) is a worthwhile problem to solve. Whether that be by PEP 325, 288, 325+288, etc., that should be discussed. Whether people use it as a pseudo-block, or decide that blocks are further worthwhile, I suppose we could wait and see.

Agreed.

I agree, in fact I think that solving that issue is very important before/if ever introducing a generalized block statement because otherwise things that would naturally be expressible with for and generators will use the block construct which allow more variety and so possibly less immediate clarity just because generators are not good at resource handling.

On 4/20/05, M.-A. Lemburg <mal@egenix.com> wrote:

Fredrik Lundh wrote:

...

PS. a side effect of the for-in pattern is that I'm beginning to feel that Python might need a nice "switch" statement based on dictionary lookups, so I can replace multiple callbacks with a single loop body, without writing too many if/elif clauses.

PEP 275 anyone ? (http://www.python.org/peps/pep-0275.html)

My use case for switch is that of a parser switching on tokens.

mxTextTools applications would greatly benefit from being able to branch on tokens quickly. Currently, there's only callbacks, dict-to-method branching or long if-elif-elif-...-elif-else.

I think "match" from Ocaml would be a much nicer addition to Python than "switch" from C. -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through.

I can post an alternative, inspired by this bit of Haskell (I've deliberately left out the Haskell type annotation for this): zoneOpts argv = case getOpt Permute options argv of (o,n,[]) -> return (o,n) (_,_,errs) -> error errs which could, in a future Python, look something like: def zoneOpts(argv): case i of getopt(argv, options, longoptions): i[2]: raise OptionError(i[2]) True: return i[:2] The intent is that within the case, the bit before each : is a boolean expression, they're evaluated in order, and the following block is executed for the first one that evaluates to be True. I know we have exceptions for this specific example, but it's just an example. I'm also assuming for the time being that getopt returns a 3-tuple (options, arguments, errors) like the Haskell version does, just for the sake of argument, and there's an OptionError constructor that will do something with that error list.. Yes, that is very different semantics from a Haskell case expression, but it kind of looks like a related idea. A more closely related idea would be to borrow the Haskell patterns: def zoneOpts(argv): case getopt(argv, options, longoptions): (o,n,[]): return o,n (_,_,errs): raise OptionError(errs) where _ matches anything, a presently unbound name is bound for the following block by mentioning it, a bound name would match whatever value it referred to, and a literal matches only itself. The first matching block gets executed. Come to think of it, it should be possible to do both. Not knowing Ocaml, I'd have to presume that 'match' is somewhat similar. Andrew On 21/04/2005, at 9:30 PM, Michael Hudson wrote:

Shannon -jj Behrens <jjinux@gmail.com> writes:

...

On 4/20/05, M.-A. Lemburg <mal@egenix.com> wrote:

...

My use case for switch is that of a parser switching on tokens.

mxTextTools applications would greatly benefit from being able to branch on tokens quickly. Currently, there's only callbacks, dict-to-method branching or long if-elif-elif-...-elif-else.

I think "match" from Ocaml would be a much nicer addition to Python than "switch" from C.

Can you post a quick summary of how you think this would work?

Cheers, mwh

-- We did requirements and task analysis, iterative design, and user testing. You'd almost think programming languages were an interface between people and computers. -- Steven Pemberton (one of the designers of Python's direct ancestor ABC) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ andrew%40indranet.co.nz

Samuele Pedroni <pedronis@strakt.com> writes:

Michael Hudson wrote:

[pattern matching]

...

Can you post a quick summary of how you think this would work?

Well, Python lists are used more imperatively and are not made up with cons cells, we have dictionaries which because of ordering issues are not trivial to match, and no general ordered records with labels.

That's a better way of putting it than "pattern matching and python don't really seem to fit together", for sure :) (I'd quite like records with labels, tangentially, but am not so wild about ordering)

We have objects and not algebraic data types. Literature on the topic usually indicates the visitor pattern as the moral equivalent of pattern matching in an OO-context vs. algebraic data types/functional one. I agree with that point of view and Python has idioms for the visitor pattern.

But the visitor pattern is pretty grim, really. It would be nice (tm) to have something like: match node in: Assign(lhs=Var(_), rhs=_): # lhs, rhs bound in here Assign(lhs=Subscr(_,_), rhs=_): # ditto Assign(lhs=Slice(*_), rhs=_): # ditto Assign(lhs=_, rhs=_): raise SyntaxError in Lib/compiler. Vyper had something like this, I think.

Interestingly even in the context of objects one can leverage the infrastructure that is there for generalized copying/pickling to allow generalized pattern matching of nested object data structures. Whether it is practical I don't know.

...

...

...

class Pt: ... def __init__(self, x,y): ... self.x = x ... self.y = y ... p(lambda _: Pt(1, _()) ).match(Pt(1,3)) (3,) p(lambda _: Pt(1, Pt(_(),_()))).match(Pt(1,Pt(Pt(5,6),3))) (<__main__.Pt instance at 0x40200b4c>, 3)

http://codespeak.net/svn/user/pedronis/match.py is an experiment in that direction (preceding this discussion and inspired while reading a book that was using OCaml for its examples).

Yikes!

Notice that this is quite grossly subclassing pickling infrastracture (the innocent bystander should probably not try that), a cleaner approach redoing that logic with matching in mind is possible and would be preferable.

Also, the syntax is disgusting. But that's a separate issue, I guess. Cheers, mwh -- /* I'd just like to take this moment to point out that C has all the expressive power of two dixie cups and a string. */ -- Jamie Zawinski from the xkeycaps source

On 4/21/05, Michael Hudson <mwh@python.net> wrote:

Shannon -jj Behrens <jjinux@gmail.com> writes:

...

On 4/20/05, M.-A. Lemburg <mal@egenix.com> wrote:

...

My use case for switch is that of a parser switching on tokens.

mxTextTools applications would greatly benefit from being able to branch on tokens quickly. Currently, there's only callbacks, dict-to-method branching or long if-elif-elif-...-elif-else.

I think "match" from Ocaml would be a much nicer addition to Python than "switch" from C.

Can you post a quick summary of how you think this would work?

Sure. Now that I'm actually trying to come up with an example, I'm noticing that Ocaml is very different than Python because Python distinguishes statements and expressions, unlike say, Scheme. Furthermore, it's important to minimize the number of new keywords and avoid excessive punctuation (which Ocaml is full of). Hence, I propose something like: def handle_token(token): match token: NUMBER: return number / a WHITESPACE if token.value == "\n": return NEWLINE (a, b): return a / b else: return token Hence, the syntax is something like (in pseudo EBNF): 'match' expr ':' {match_expression ':' block}* 'else' ':' block match_expr ::= lvalue | constant_expression Sematically, the above example translates into: def handle_token(token): if token == NUMBER: return number / a elif token == WHITESPACE and token.value == "\n": return NEWLINE elif "setting (a, b) = token succeeds": return a / b else: return token However, unlike the code above, you can more easily and more aggressively optimize. Best Regards, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through.

On 4/25/05, M.-A. Lemburg <mal@egenix.com> wrote:

Shannon -jj Behrens wrote:

...

On 4/20/05, M.-A. Lemburg <mal@egenix.com> wrote:

...

Fredrik Lundh wrote:

...

PS. a side effect of the for-in pattern is that I'm beginning to feel that Python might need a nice "switch" statement based on dictionary lookups, so I can replace multiple callbacks with a single loop body, without writing too many if/elif clauses.

PEP 275 anyone ? (http://www.python.org/peps/pep-0275.html)

My use case for switch is that of a parser switching on tokens.

mxTextTools applications would greatly benefit from being able to branch on tokens quickly. Currently, there's only callbacks, dict-to-method branching or long if-elif-elif-...-elif-else.

I think "match" from Ocaml would be a much nicer addition to Python than "switch" from C.

PEP 275 is about branching based on dictionary lookups which is somewhat different than pattern matching - for which we already have lots and lots of different tools.

The motivation behind the switch statement idea is that of interpreting the multi-state outcome of some analysis that you perform on data. The main benefit is avoiding Python function calls which are very slow compared to branching to inlined Python code.

Having a simple switch statement would enable writing very fast parsers in Python - you'd let one of the existing tokenizers such as mxTextTools, re or one of the xml libs create the token input data and then work on the result using a switch statement.

Instead of having one function call per token, you'd only have a single dict lookup.

BTW, has anyone in this thread actually read the PEP 275 ?

I'll admit that I haven't because dict-based lookups aren't as interesting to me as an Ocaml-style match statement. Furthermore, the argument "Instead of having one function call per token, you'd only have a single dict lookup" isn't very compelling to me personally, because I don't have such a performance problem in my applications, which isn't to say that it isn't important or that you don't have a valid point. Best Regards, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through.

On Apr 19, 2005, at 15:57, BJörn Lindqvist wrote:

...

RSMotD (random stupid musing of the day): so I wonder if the decorator syntax couldn't be extended for this kind of thing.

@acquire(myLock): code code code

Would it be useful for anything other than mutex-locking? And wouldn't

Well, one obvious use might be, say: @withfile('foo.bar', 'r'): content = thefile.read() but that would require the decorator and block to be able to interact in some way, so that inside the block 'thefile' is defined suitably.

it be better to make a function of the block wrapped in a block-decorator and then use a normal decorator?

From a viewpoint of namespaces, I think it would be better to have the block execute in the same namespace as the code surrounding it, not a separate one (assigning to 'content' would not work otherwise), so a nested function would not be all that useful. The problem might be, how does the _decorator_ affect that namespace. Perhaps: def withfile(filename, mode='r'): openfile = open(filename, mode) try: block(thefile=openfile) finally: openfile.close() i.e., let the block take keyword arguments to tweak its namespace (but assignments within the block should still affect its _surrounding_ namespace, it seems to me...). Alex

Steven Bethard wrote:

Of course, even with the unpack list, you still have to know what kind of arguments the function calls your block with. And because these only appear within the code, e.g. block(openfile) you can't rely on easily accessible things like the function's signature.

You can't rely on a function's signature alone to tell you much in any case. A distressingly large number of functions found in third-party extension modules have a help() string that just says something like fooble(arg,...) There's really no substitute for a good docstring! -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+

So while: fooble(arg) is pretty nasty, documentation that tells me that 'arg' is a string is probably enough to set me on the right track. But if the documentation tells me that arg is a thunk/block, that's almost certainly not enough to get me going. I also need to know how that thunk/block will be called.

This argument against thunks sounds bogus to me. The signature of any callable arguments is recursively part of the signature of the function you're documenting. Just like the element type of any sequence arguments is part of the argument type. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

In case my point about the difference between thunks and other callables (specifically decorators) slipped by, consider the documentation for staticmethod, which takes a callable. All the staticmethod documentation says about that callable's parameters is: "A static method does not receive an implicit first argument" Pretty simple I'd say. Or classmethod: "A class method receives the class as implicit first argument, just like an instance method receives the instance." Again, pretty simple. Why are these simple? Because decorators generally pass on pretty much the same arguments as the callables they wrap. My point was just that because thunks don't wrap other normal callables, they can't make such abbreviations.

You've got the special-casing backwards. It's not thinks that are special, but staticmethod (and decorators in general) because they take *any* callable. That's unusual -- most callable arguments have a definite signature, think of map(), filter(), sort() and Button callbacks. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Alex Martelli wrote:

def withfile(filename, mode='r'): openfile = open(filename, mode) try: block(thefile=openfile) finally: openfile.close()

i.e., let the block take keyword arguments to tweak its namespace

I don't think I like that idea, because it means that from the point of view of the user of withfile, the name 'thefile' magically appears in the namespace without it being obvious where it comes from.

(but assignments within the block should still affect its _surrounding_ namespace, it seems to me...).

I agree with that much. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+

On Tue, Apr 19, 2005, Shane Holloway (IEEE) wrote:

I heartily agree! Especially when you have very similar try/finally code you use in many places, and wish to refactor it into a common area. If this is done, you are forced into a callback form like follows::

def withFile(filename, callback): aFile = open(filename, 'r') try: result = callback(aFile) finally: aFile.close() return result

class Before: def readIt(self, filename): def doReading(aFile): self.readPartA(aFile) self.readPartB(aFile) self.readPartC(aFile)

withFile(filename, doReading)

Which is certainly functional. I actually use the idiom frequently. However, my opinion is that it does not read smoothly. This form requires that I say what I'm doing with something before I know the context of what that something is. For me, blocks are not about shortening the code, but rather clarifying *intent*.

Hmmmm.... How is this different from defining functions before they're called? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR

[Shane Holloway]

When I write classes, I tend to put the public methods at the top. Utility methods used by those entry points are placed toward the bottom. In this way, I read the context of what I'm doing first, and then the details of the internal methods as I need to understand them.

Granted I could achieve this effect with::

class Before: def readIt(self, filename): def readIt(): withFile(filename, doReading)

def doReading(aFile): self.readPartA(aFile) self.readPartB(aFile) self.readPartC(aFile)

return readIt()

Which is fine with me, but the *intent* is more obfuscated than what the block construct offers. And I don't think my crew would appreciate if I did this very often. ;)

I typically solve that by making doReading() a method: class Before: def readit(self, filename): withFile(filename, self._doReading) def _doReading(self, aFile): self.readPartA(aFile) self.readPartB(aFile) self.readPartC(aFile) Perhaps not as Pure, but certainly Practical. :-) And you could even use __doReading to make it absolutely clear that doReading is a local artefact, if you care about such things. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

[Greg Ewing]

My current thought is that it should look like this:

with_file(filename) as f: do_something_with(f)

The success of this hinges on how many use cases can be arranged so that the word 'as' makes sense in that position. [...] This way, the syntax is just

expr ['as' assignment_target] ':' suite

and the expr is evaluated quite normally.

Perhaps it could be even simpler: [assignment_target '=']* expr ':' suite This would just be an extension of the regular assignment statement. (More in a longer post I'm composing off-line while picking cherries off the thread.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

[Ping]

It sounds like you are very close to simply translating

expression... function_call(args): suite

into

expression... function_call(args)(suitefunc)

Actually, I'm abandinging this interpretation; see my separate (long) post. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Greg Ewing wrote:

I also have a thought concerning whether the block argument to the function should come first or last or whatever. My solution is that the function should take exactly *one* argument, which is the block. Any other arguments are dealt with by currying. In other words, with_file above would be defined as

def with_file(filename): def func(block): f = open(filename) try: block(f) finally: f.close() return func

This would also make implementation much easier. The parser isn't going to know that it's dealing with anything other than a normal expression statement until it gets to the 'as' or ':', by which time going back and radically re-interpreting a previous function call could be awkward.

I made an example implementation, and this wasn't an issue. It took some code to stick the thunk into the argument list, but it was pretty straightforward. The syntax that is actually used by the parser can be the same regardless of whether or not argument list augmentation is done, so the parser will not find one more awkward than the other.

This way, the syntax is just

expr ['as' assignment_target] ':' suite

and the expr is evaluated quite normally.

Requiring arguments other than the block to be dealt with by currying can lead to problems. I won't claim these problems are serious, but they will be annoying. Say, for example, you create a block-accepting function that takes no arguments. Naturally, you would define it like this: def f(block): do_something_with_block Now, say you want to add to this function an optional argument, so you wrap another function around it like in your 'with_file' example above. Unfortunately, now you need to go find every call of this function and add empty parentheses. This is annoying. Remember the first time you added optional arguments to a function and what a relief it was not to have to go find every call to that function and stick in the extra argument? Those days are over! (well, in this case anyway.) Some people, aware of this problem of adding optional arguments, will define *all* of their block-accepting functions so that they are wrapped in another function, even if that function takes no arguments (and wars, annoying ones, will be fought over whether this is the "right" way to do it or not!): def f(): def real_func(block): pass return real_func Now the documentation gets confusing. Just saying that the function doesn't take any non-block arguments isn't enough. You would need very specific language, which many library authors will not provide. And there will always be that extra step in thought: do I need the stupid parentheses or not? There will inevitably be people (including me) who get the parentheses wrong because of absentmindedness or carelessness. This will be an extra little speed bump. Now, you may say that all these arguments apply to function decorators, so why have none of these problems appeared? The difference is that defining a function takes a long time, so a little speed bump when decorating it isn't a big deal. But blocks can be defined almost instantly. Much of their purpose has to do with making things quicker. Speed bumps are therefore a bigger deal. This will also be an issue for beginners who use python. A beginner won't necessarily have a good understanding of a function that returns a function. But such an understanding would be required simply to *use* block-accepting functions. Otherwise it would be completely mysterious why sometimes one sees this f(a,b,c) as i: pass and sometimes this g as i: pass even though both of these cases just seem to call the function that appears next to 'as' (imagine you don't have the source of 'f' and 'g'). Even worse, imagine finally learning the rule that parentheses are not allowed if there are zero arguments, and then seeing: h() as i: pass Now it would just seem arbitrary whether or not parentheses are required or disallowed. Such an issue may seem trivial to an experienced programmer, but can be very off-putting for a beginner.

...

Another set of question arose for me when Barry started musing over the combination of blocks and decorators. What are blocks? Well, obviously they are callable. What do they return? The local namespace they created/modified?

I think the return value of a block should be None. In constructs like with_file, the block is being used for its side effect, not to compute a value for consumption by the block function. I don't see a great need for blocks to be able to return values.

If you google "filetype:rb yield", you can see many the uses of yield in ruby. By looking for the uses in which yield's return value is used, you can find blocks that return values. For example, "t = yield()" or "unless yield()" indicate that a block is returning a value. It is true that most of the time blocks do not return values, but I estimate that maybe 20% of the hits returned by google contain at least one block that does. Of course, this information is alone is not very informative, one would like to understand each case individually. But, as a first guess, it seems that people do find good uses for being able to return a value from a block. Probably 'continue <block_retun_val>', which I had proposed earlier, is awful syntax for returning a value from a block. But 'produce <block_return_val>' or some other verb may not be so bad. In cases that the block returns no value, 'continue' could still be used to indicate that control should return to the function that called the block. -Brian

On Tuesday 19 Apr 2005 20:24, Guido van Rossum wrote: ..

*If* we're going to create syntax for anonymous blocks, I think the primary use case ought to be cleanup operations to replace try/finally blocks for locking and similar things. I'd love to have syntactical support so I can write

blahblah(myLock): code code code

I've got a basic parser that I wrote last summer which was an experiment in a generic "python-esque" parser. It might be useful for playing with these thing since it accepted the above syntax without change, among many others, happily. (Added as a 38th test, and probably the sixth "language syntax" it understands) It's also entirely keyword free, which strikes me as a novelty. The abstract syntax tree that's generated for it is rather unwieldy and over the top, but that's as far as the parser goes. (As I said I was interested in a generic parser, not a language :) The (entire) grammar resulting was essentially this: (and is LR-parsable) program -> block block -> BLOCKSTART statement_list BLOCKEND statement_list -> statement* statement -> (expression | expression ASSIGNMENT expression | ) EOL expression -> oldexpression (COMMA expression)* oldexpression -> (factor [factorlist] | factor INFIXOPERATOR expression ) factorlist -> factor* factor factor -> ( bracketedexpression | constructorexpression | NUMBER | STRING | ID | factor DOT dotexpression | factor trailer | factor trailertoo ) dotexpression -> (ID bracketedexpression | factor ) bracketedexpression -> BRA [ expression ] KET constructorexpression -> BRA3 [ expression ] KET3 trailer -> BRA2 expression KET2 trailertoo -> COLON EOL block The parser.out file for the curious is here: * http://www.cerenity.org/SWP/parser.out (31 productions) The parser uses a slightly modified PLY based parser and might be useful for playing around with constructs (Might not, but it's the reason I'm mentioning it). The approach taken is to treat ":" as always starting a code block to be passed. The first token on the line is treated as a function name. The idea was that "def", "class", "if", etc then become simple function calls that get various arguments which may include one or more code blocks. The parser was also written entirely test first (as an experiment to see what that's like for writing a parser) and includes a variety of sample programs that pass. (39 different program tests) I've put a tarball here: * http://www.cerenity.org/SWP-0.0.0.tar.gz (includes the modifed version of PLY) * Also browesable here: http://www.cerenity.org/SWP/ * Some fun examples: * Python-like http://www.cerenity.org/SWP/progs/expr_29.p (this example is an earlier version of the parser) * LOGO like: http://www.cerenity.org/SWP/progs/expr_33.p * L-System definition: http://www.cerenity.org/SWP/progs/expr_34.p * SML-like: http://www.cerenity.org/SWP/progs/expr_35.p * Amiga E/Algol like: http://www.cerenity.org/SWP/progs/expr_37.p Needs the modified version of PLY installed first, and the tests can be run using "runtests.sh". Provided in case people want to play around with something, I'm happy with the language as it is. :-) Best Regards, Michael, -- Michael Sparks, Senior R&D Engineer, Digital Media Group Michael.Sparks@rd.bbc.co.uk, British Broadcasting Corporation, Research and Development Kingswood Warren, Surrey KT20 6NP This e-mail may contain personal views which are not the views of the BBC.

Fredrik Lundh wrote:

...

In short, if doFoo is defined like:

def doFoo(func1, func2): pass

You would be able to call it like:

doFoo(**): def func1(a, b): return a + b def func2(c, d): return c + d

That is, a suite can be used to define keyword arguments.

umm. isn't that just an incredibly obscure way to write

def func1(a, b): return a + b def func2(c, d): return c + d doFoo(func1, func2)

but with more indentation?

If suites were commonly used as above to define properties, event handlers and other callbacks, then I think most people would be able to comprehend what the first example above is doing much more quickly than the second. So, I don't find it obscure for any reason other than because no one does it. Also, the two examples above are not exactly the same since the two functions are defined in a separate namespace in the top example. -Brian

Fredrik Lundh wrote:

Brian Sabbey wrote:

...

If suites were commonly used as above to define properties, event handlers and other callbacks, then I think most people would be able to comprehend what the first example above is doing much more quickly than the second.

wonderful logic, there. good luck with your future adventures in language design.

</F>

I'm just trying to help python improve. Maybe I'm not doing a very good job, I don't know. Either way, there's no need to be rude. If I've broken some sort of unspoken code of behavior for this list, then maybe it would be easier if you just 'spoke' it (perhaps in a private email or in the description of this list on python.org). I'm not sure what your point is exactly. Are you saying that any language feature that needs to be commonly used to be comprehendible will never be comprehendible because it will never be commonly used? If so, then I do not think you have a valid point. I never claimed that keyword suites *need* to be commonly used to be comprehendible. I only said that if they were commonly used they would be more comprehendible than the alternative. I happen to also believe that seeing them once or twice is enough to make them about equally as comprehendible as the alternative. -Brian

Fredrik Lundh wrote:

Brian Sabbey wrote:

...

doFoo(**): def func1(a, b): return a + b def func2(c, d): return c + d

That is, a suite can be used to define keyword arguments.

umm. isn't that just an incredibly obscure way to write

def func1(a, b): return a + b def func2(c, d): return c + d doFoo(func1, func2)

but with more indentation?

Brian's suggestion makes the code read more like an outline. In Brian's example, the high-level intent stands out from the details, while in your example, there is no visual cue that distinguishes the details from the intent. Of course, lambdas are even better, when it's possible to use them: doFoo((lambda a, b: a + b), (lambda c, d: c + d)) Shane