From janssen at parc.com Thu Sep 4 03:50:26 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 3 Sep 2008 18:50:26 PDT Subject: [Python-ideas] More power in list comprehensions with the 'as' keyword In-Reply-To: <47c890dc0808271629v3b5f5713id517567d7a68ae99@mail.gmail.com> References: <94bdd2610808270951g2c191469g97e5008775c3de01@mail.gmail.com> <47c890dc0808271629v3b5f5713id517567d7a68ae99@mail.gmail.com> Message-ID: <08Sep3.185035pdt."58698"@synergy1.parc.xerox.com> > Indeed, probably something approaching Common Lisp's overcomplicated > "loop" macro: > http://www.unixuser.org/~euske/doc/cl/loop.html Hey, "loop" is a thing of beauty and a joy forever! Or are you proposing macros for Python? Bill From janssen at parc.com Thu Sep 4 04:00:43 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 3 Sep 2008 19:00:43 PDT Subject: [Python-ideas] More power in list comprehensions with the 'as' keyword In-Reply-To: References: <94bdd2610808270951g2c191469g97e5008775c3de01@mail.gmail.com> <3f4107910808280204jc24447bjaa52151bbd1f6d0b@mail.gmail.com> <48B6D035.6080804@gmx.net> Message-ID: <08Sep3.190051pdt."58698"@synergy1.parc.xerox.com> > > We already a "with Expression as Identifier" syntax that is well known > > and used in Python: why use something different? > > > > [stripped for l in text.split('\n') with l.strip() as stripped if stripped != ''] > > > > will be a much better syntax to parse and acquire for a typical > > pythonista. ;) > > Just because it exists, doesn't mean that it's "well known and used". > Also, don't conflate the need to handle context management (locking, > closing files, etc.) with the false perceived need to add temporary > assignments in list comprehensions and generator expressions. Hmmm, there's also "import foo as bar" and "from foo import bar as bletch". So "as" is gaining some traction here for these purposes (binding a value to a named symbol). That being said, I also prefer [x for x in (x.strip() for x in t.split(pat)) if x] Bill From dangyogi at gmail.com Thu Sep 4 16:58:20 2008 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Thu, 04 Sep 2008 10:58:20 -0400 Subject: [Python-ideas] should there be a difference between generators and iterators? Message-ID: <48BFF78C.3070102@gmail.com> There are several points here that might or might not be adopted individually. The specific prompt for this message is that I've run into a snag using generators with 'itertools.chain.from_iterable'. I need to be able to control when the generators called by 'chain' get closed (so that their 'finally' clauses are run). Unfortunately, chain does not propagate 'close' (or 'throw' or 'send') back to the generators that it's using. Nor, for that matter do any of the other tools in itertools, or the builtin map function (to my knowledge). I propose the following. Each of these is independent of the others, but all related to cleaning up how this works: 1. All of the itertools and map (and I've no doubt left some others out here) be extended to propagate the extra generators methods: close, throw and send. 2. That the 'for' loop be extended so that if an exception occurs within its body, it calls 'throw' on its iterable (if it has a throw method). 3. That the 'for' loop be extended to call the 'close' method on its iterable (if it has a close method) when the loop terminates (either normally, with break, or with an exception). 4. That a 'not_closing' builtin function be added that takes an iterable and shields it from a 'close' call. 5. That 'close' and 'throw' be added to all iterables. Motivation: The motivation for each proposal is (by their number): 1. This is the one that I'm specifically stuck on. I was relying on garbage collection to do this, but this doesn't work in jython and ironpython... Since the chain function is dealing with two iterables (an inner and outer iterable), I think that it makes sense for it to check whether each of these have the extra methods or not. For example, the inner iterable may have a 'close', but not the outer iterable (or vise versa). This shouldn't cause an error if 'close' is called on the chain. This one, specifically, would be helpful for me to move to Python 3K; so the sooner the better! (Please!) 2. There has been some discussion here about extending 'for' loops that has touched on non-local continue/break capability. If step 2 is provided, this capability could be provided as follows: class funky: ... def continue_(self): raise ContinueError(self) def break_(self): raise BreakError(self) def throw(self, type, value, tb): if issubclass(type, ContinueError) and value.who is self: return next(self) if issubclass(type, BreakError) and value.who is self: raise StopIteration top = funky(iterable1) for x in top: middle = funky(iterable2) for y in middle: bottom = funky(iterable3) for z in bottom: ... middle.continue_() 3. But this is a problem with: for line in filex: if test1(line): break for line in filex: ... which brings us to: 4. A solution to the above: for line in not_closing(filex): if test1(line): break for line in filex: ... 5. I thought that I may as well throw this in for discussion... This might cause some consternation to those who has written their own iterables... From python at rcn.com Thu Sep 4 17:15:13 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 4 Sep 2008 08:15:13 -0700 Subject: [Python-ideas] should there be a difference between generators anditerators? References: <48BFF78C.3070102@gmail.com> Message-ID: From: "Bruce Frederiksen" > 1. All of the itertools and map (and I've no doubt left some others out > here) be extended to propagate the extra generators methods: close, > throw and send. Try wrapping each input iterator with an autocloser: def closer(it): for elem in it: yield elem if hasattr(it, 'close'): it.close() map(somefunc, closer(someiterable)) Raymond From dangyogi at gmail.com Thu Sep 4 18:30:25 2008 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Thu, 04 Sep 2008 12:30:25 -0400 Subject: [Python-ideas] should there be a difference between generators anditerators? In-Reply-To: References: <48BFF78C.3070102@gmail.com> Message-ID: <48C00D21.3020802@gmail.com> Raymond Hettinger wrote: > From: "Bruce Frederiksen" >> 1. All of the itertools and map (and I've no doubt left some others >> out here) be extended to propagate the extra generators methods: >> close, throw and send. > > Try wrapping each input iterator with an autocloser: > > def closer(it): > for elem in it: > yield elem > if hasattr(it, 'close'): > it.close() > > map(somefunc, closer(someiterable)) I don't think that this adds anything. If the 'it' iterable is run to completion (and, presumably, does what 'close' will do anyway), this adds nothing. And if the 'it' iterable is not run to completion, then neither is closer, and so it still adds nothing. (And placing the 'if hasattr' clause under a 'finally' doesn't help, because the 'finally' is only run if the generator runs to completion or 'close' is called, and nobody is ever going to call close here on closer). What I'm looking for is making the following work: with contextlib.closing(map(somefunc, someiterable)) as it: for i in it: if somecondition: break # at this point someiterable.close should have been run Intuitively, I would think that the above should work, but it doesn't. Intuitively, I would think that if Python defines 'close' and 'throw' methods to allow generators to properly clean up after themselves, that itertools and the 'for' statement would both honor these so that the 'with' statement isn't even required here. I can see arguments for not calling 'close' automatically in 'for' statements (though none for not calling 'throw' automatically); but I don't see any arguments against itertools honoring these methods. If any of the itertools is passing values back from a generator (rather than a simple iterator), it would be very nice to retain the full semantics of the generator. Now, for map, you could suggest that Python programmers understand these subtleties and write instead: with contextlib.closing(someiterable) as it: for i in map(somefunc, it): and this works for map. But what about chain? with contextlib.closing(chain.from_iterable(map(somegenerator, someiterable))) as it: for i in it: This is my specific situation, and I need 'close' to be called on the currently active somegenerator within chain before the line following 'with' is executed. In my particular case, I don't think I need close on someiterable, but it seems to make sense to do this too. Currently, I'm not using the 'with' clause and am relying on CPython's garbage collector to immediately call __del__ (which is defined to call close) when the 'for' statement abandons chain. But this doesn't work in jython or ironpython... -bruce And, BTW, thank you for adding from_iterable to chain! From dangyogi at gmail.com Fri Sep 5 01:01:53 2008 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Thu, 4 Sep 2008 19:01:53 -0400 Subject: [Python-ideas] should there be a difference between generators and iterators? In-Reply-To: <48BFF78C.3070102@gmail.com> References: <48BFF78C.3070102@gmail.com> Message-ID: <7d702edf0809041601j7a23adc6r104c7028a721271e@mail.gmail.com> I guess while we're at it, I'd like to add the following to the discussion: 6. Add the context manager methods (__enter__ and __exit__) to generators and, by extension, to itertools. This makes it easier to use 'with' to get the clean up ('finally' clauses) done. On Thu, Sep 4, 2008 at 10:58 AM, Bruce Frederiksen wrote: > There are several points here that might or might not be adopted > individually. > > [...] > > I propose the following. Each of these is independent of the others, but > all related to cleaning up how this works: > > 1. All of the itertools and map (and I've no doubt left some others out > here) be extended to propagate the extra generators methods: close, throw > and send. > > 2. That the 'for' loop be extended so that if an exception occurs within > its body, it calls 'throw' on its iterable (if it has a throw method). > > 3. That the 'for' loop be extended to call the 'close' method on its > iterable (if it has a close method) when the loop terminates (either > normally, with break, or with an exception). > > 4. That a 'not_closing' builtin function be added that takes an iterable > and shields it from a 'close' call. > > 5. That 'close' and 'throw' be added to all iterables. From guido at python.org Fri Sep 5 01:29:20 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 4 Sep 2008 16:29:20 -0700 Subject: [Python-ideas] should there be a difference between generators and iterators? In-Reply-To: <48BFF78C.3070102@gmail.com> References: <48BFF78C.3070102@gmail.com> Message-ID: The proposal to extend all iterators with an additional protocol goes directly against one of the design goals for iterators, i.e., that it should be easy to implement a new iterator, without subclassing something. So any proposal that wants to add new methods to *all* iterators (let alone all iterables, which is a much larger set -- read abc.py in the 3.0 stdlib for the difference) is doomed. That said, having *optional* additions to the protocol might still be reasonably debated. On the 3rd hand, I'd like to understand more about your use case. On Thu, Sep 4, 2008 at 7:58 AM, Bruce Frederiksen wrote: > There are several points here that might or might not be adopted > individually. > > The specific prompt for this message is that I've run into a snag using > generators with 'itertools.chain.from_iterable'. I need to be able to > control when the generators called by 'chain' get closed (so that their > 'finally' clauses are run). > > Unfortunately, chain does not propagate 'close' (or 'throw' or 'send') back > to the generators that it's using. > > Nor, for that matter do any of the other tools in itertools, or the builtin > map function (to my knowledge). > > I propose the following. Each of these is independent of the others, but > all related to cleaning up how this works: > > 1. All of the itertools and map (and I've no doubt left some others out > here) be extended to propagate the extra generators methods: close, throw > and send. > > 2. That the 'for' loop be extended so that if an exception occurs within > its body, it calls 'throw' on its iterable (if it has a throw method). > > 3. That the 'for' loop be extended to call the 'close' method on its > iterable (if it has a close method) when the loop terminates (either > normally, with break, or with an exception). > > 4. That a 'not_closing' builtin function be added that takes an iterable > and shields it from a 'close' call. > > 5. That 'close' and 'throw' be added to all iterables. > > > Motivation: > > The motivation for each proposal is (by their number): > > 1. This is the one that I'm specifically stuck on. I was relying on > garbage collection to do this, but this doesn't work in jython and > ironpython... Since the chain function is dealing with two iterables (an > inner and outer iterable), I think that it makes sense for it to check > whether each of these have the extra methods or not. For example, the inner > iterable may have a 'close', but not the outer iterable (or vise versa). > This shouldn't cause an error if 'close' is called on the chain. > > This one, specifically, would be helpful for me to move to Python 3K; so the > sooner the better! (Please!) > > 2. There has been some discussion here about extending 'for' loops that has > touched on non-local continue/break capability. If step 2 is provided, this > capability could be provided as follows: > > class funky: > ... > def continue_(self): raise ContinueError(self) > def break_(self): raise BreakError(self) > def throw(self, type, value, tb): > if issubclass(type, ContinueError) and value.who is self: > return next(self) > if issubclass(type, BreakError) and value.who is self: > raise StopIteration > > top = funky(iterable1) > for x in top: > middle = funky(iterable2) > for y in middle: > bottom = funky(iterable3) > for z in bottom: > ... > middle.continue_() > > 3. But this is a problem with: > > for line in filex: > if test1(line): break > for line in filex: > ... > > which brings us to: > > 4. A solution to the above: > > for line in not_closing(filex): > if test1(line): break > for line in filex: > ... > > 5. I thought that I may as well throw this in for discussion... This might > cause some consternation to those who has written their own iterables... > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bruce at leapyear.org Fri Sep 5 08:43:06 2008 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 4 Sep 2008 23:43:06 -0700 Subject: [Python-ideas] should there be a difference between generators and iterators? In-Reply-To: <48BFF78C.3070102@gmail.com> References: <48BFF78C.3070102@gmail.com> Message-ID: I think it makes sense that itertools should pass on throw, etc. I'd be interested in whether anything would break on this change. I don't think I like the other suggestions. Having an exception in some random part of a loop throw *into* the iterator of the loop, just seems weird. For the examples, you give, couldn't break_ do the throw itself? for line in filex: > if test1(line): break > for line in filex: > ... > This example convinces me that 3 could introduce untold chaos into existing code. Wouldn't something like this make more sense? for line in filex: if test1(line): break finally: filex.close() (And yes, I know the time machine can do that with one extra word and perhaps with handles this better.) To get __enter__ and __exit__ behavior for an iterator, can't you just wrap it in class that provides that capability and calls close? You might need itertools to have some support that extended iterator class but that seems simpler. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangyogi at gmail.com Fri Sep 5 15:08:02 2008 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Fri, 05 Sep 2008 09:08:02 -0400 Subject: [Python-ideas] should there be a difference between generators and iterators? In-Reply-To: References: <48BFF78C.3070102@gmail.com> Message-ID: <48C12F32.5040405@gmail.com> OK, so point 5 is out. That leaves the other 4 points... :-) My specific use case involves using generators to establish variable bindings prior to each yield. These bindings are then undone after the yield in preparation for the next iteration. When the generator is finished, no bindings should remain. This is basically a "shallow binding" scheme (from the lisp community) using generators. So I have a "finally" clause in the generator to clean things up. And then, to complicate matters, I am not just interested in the output of a single application of this generator, but am interested in the output applied successively to the items in a tuple (the generator applied to each item yielding several times). So I use 'chain' from itertools, with the 'from_iterable' option (which is greatly appreciated!): for y in chain.from_iterable(gen(x) for x in sometuple): ... Now, it's possible that the for loop executes a 'break' or that an exception will terminate the loop prematurely. But I still need the bindings undone by the last yield from 'gen' (i.e., I need its finally clause run). Currently, when the 'for' loop terminates on CPython, __del__ is immediately called, which calls close, which runs the finally clause, and all is well! But on jython and ironpython (both of which are close to releasing a 2.5 compatible version with these extra generator functions in them for the first time), that doesn't work and never will. So I need to fix this to not rely on the garbage collector. I can't see any way to force the 'finally' clause to run, given that the last used 'gen' is hidden inside chain. This is what prompted my point #1. It would be nice, in more general terms, if using itertools didn't hide the extra behavior that generators provide. It would be nice if I could still use the additional generator methods on the objects returned by the itertools and by map if I give them a generator as input. Then I could do: with closing(chain.from_iterable(gen(x) for x in sometuple)) as g: for y in g: ... OK, so moving up another level (getting to point #2), what is happening is that the 'finally' clause in a generator isn't being honored by 'for' statements. So 'finally' doesn't really mean much in generators (unless you're running on CPython, in which case the garbage collector covers you): for x in gen(y): ... break # doesn't run finally in gen on jython or ironpython! So if point #2 where also adopted (and since 'throw' also runs the 'finally' clause in the generator) and, if 'break' sends GeneratorExit to 'for's iterable (via 'throw') and then ignores the exception when it comes back from the 'throw'; then the 'with' should never be required! In this case, the 'for' statement would completely honor the 'finally' clause of generators passed to it regardless of loop termination, and 'finally' for generators would mean finally (like it does everywhere else) -- even in jython and ironpython. Adopting point #2 does basically prohibit the straightforward use of the same generator in multiple 'for' loops: g = gen(x) for i in g: ... break for j in g: ... But the only real use case that I can see for this is with files, which don't have a 'throw' method, so point #2 doesn't break that use case. A wrapper that shields the generator from 'throw' would allow the use case above to still be done. I'd don't see any "gotchas" implementing point #1 and a remote gotcha on point #2. And I see these first two points as the important ones, because they fix a "bug" in the current definition of the language. But adopting point #1 alone allows me to fix my specific problem using 'with closing'. I do see gotchas with point 3, assuming that the above is already an established use case with files (hence point 4). But I threw them out anyway for discussion. I don't really expect that they'll be adopted, unless somebody else sees something in them, or some other way around the problem of multiple use of the same iterable. If point #2 is not adopted, adding the context manager methods to generators (like files have) would be a nice touch too! (point #6 in a later post). I hope all of this helps! -bruce Guido van Rossum wrote: > The proposal to extend all iterators with an additional protocol goes > directly against one of the design goals for iterators, i.e., that it > should be easy to implement a new iterator, without subclassing > something. So any proposal that wants to add new methods to *all* > iterators (let alone all iterables, which is a much larger set -- read > abc.py in the 3.0 stdlib for the difference) is doomed. That said, > having *optional* additions to the protocol might still be reasonably > debated. On the 3rd hand, I'd like to understand more about your use > case. > > On Thu, Sep 4, 2008 at 7:58 AM, Bruce Frederiksen wrote: > >> There are several points here that might or might not be adopted >> individually. >> >> The specific prompt for this message is that I've run into a snag using >> generators with 'itertools.chain.from_iterable'. I need to be able to >> control when the generators called by 'chain' get closed (so that their >> 'finally' clauses are run). >> >> Unfortunately, chain does not propagate 'close' (or 'throw' or 'send') back >> to the generators that it's using. >> >> Nor, for that matter do any of the other tools in itertools, or the builtin >> map function (to my knowledge). >> >> I propose the following. Each of these is independent of the others, but >> all related to cleaning up how this works: >> >> 1. All of the itertools and map (and I've no doubt left some others out >> here) be extended to propagate the extra generators methods: close, throw >> and send. >> >> 2. That the 'for' loop be extended so that if an exception occurs within >> its body, it calls 'throw' on its iterable (if it has a throw method). >> >> 3. That the 'for' loop be extended to call the 'close' method on its >> iterable (if it has a close method) when the loop terminates (either >> normally, with break, or with an exception). >> >> 4. That a 'not_closing' builtin function be added that takes an iterable >> and shields it from a 'close' call. >> >> 5. That 'close' and 'throw' be added to all iterables. >> >> >> Motivation: >> >> The motivation for each proposal is (by their number): >> >> 1. This is the one that I'm specifically stuck on. I was relying on >> garbage collection to do this, but this doesn't work in jython and >> ironpython... Since the chain function is dealing with two iterables (an >> inner and outer iterable), I think that it makes sense for it to check >> whether each of these have the extra methods or not. For example, the inner >> iterable may have a 'close', but not the outer iterable (or vise versa). >> This shouldn't cause an error if 'close' is called on the chain. >> >> This one, specifically, would be helpful for me to move to Python 3K; so the >> sooner the better! (Please!) >> >> 2. There has been some discussion here about extending 'for' loops that has >> touched on non-local continue/break capability. If step 2 is provided, this >> capability could be provided as follows: >> >> class funky: >> ... >> def continue_(self): raise ContinueError(self) >> def break_(self): raise BreakError(self) >> def throw(self, type, value, tb): >> if issubclass(type, ContinueError) and value.who is self: >> return next(self) >> if issubclass(type, BreakError) and value.who is self: >> raise StopIteration >> >> top = funky(iterable1) >> for x in top: >> middle = funky(iterable2) >> for y in middle: >> bottom = funky(iterable3) >> for z in bottom: >> ... >> middle.continue_() >> >> 3. But this is a problem with: >> >> for line in filex: >> if test1(line): break >> for line in filex: >> ... >> >> which brings us to: >> >> 4. A solution to the above: >> >> for line in not_closing(filex): >> if test1(line): break >> for line in filex: >> ... >> >> 5. I thought that I may as well throw this in for discussion... This might >> cause some consternation to those who has written their own iterables... >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> >> > > > > From dangyogi at gmail.com Fri Sep 5 17:24:14 2008 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Fri, 05 Sep 2008 11:24:14 -0400 Subject: [Python-ideas] should there be a difference between generators and iterators? In-Reply-To: References: <48BFF78C.3070102@gmail.com> Message-ID: <48C14F1E.20802@gmail.com> Bruce Leban wrote: > > I don't think I like the other suggestions. Having an exception in > some random part of a loop throw *into* the iterator of the loop, just > seems weird. For the examples, you give, couldn't break_ do the throw > itself? First, the easy part. Yes break could also do a throw (see my response to Guido). There are two ways to try to visualize this. First explanation: A function, say 'bar', has input and produces output. When it raises an exception, there could be two reasons for this: 1. It doesn't like its input (in which case the input might be fixed and new values provided), or 2. It's unable to produce its output. With traditional functions, the caller of 'bar' is responsible for providing its input and also receives its output: input = foo(...) output = bar(input) or, simply: output = bar(foo(...)) Since the function called to produce the input ('foo') is no longer around when 'bar' is called, the distinction above hasn't been important, because either way the caller has to deal with the problem. In the first case, the caller might produce some other input value and call 'bar' again. In the second case, the caller must proceed without the output. But when generators provide input to a function, the generator is still around when the function is run. So it makes sense, in the first case, to raise the exception in the generator and give it a chance to fix the input value. And this is exactly how the new (in 2.5) 'throw' method is defined to act on the generator side (in PEP 342). If we knew which exceptions meant "bad input" vs "output not possible", we could only raise the first kind in the generator. But we don't know this. So it makes sense to first raise all exceptions on the input side in the generator. If the generator recognizes the exception (i.e., as an 'input error' exception) and can fix the problem, then 'bar' may still be able to produce output. If not, then forward the exception on to the output side of 'bar' (as an 'output not possible' exception). Applying this logic to the 'for' statement is what leads to my point #2: for input in foo(...): output = bar(input) If 'bar' raises an exception, it should first go to 'foo' (if 'foo' has a 'throw' method), and then to the outer block containing the 'for' statement. If the generator's 'throw' method returns a value, then the 'for' statement would assign this value to 'input' and run its body again, proceeding normally (the exception has been taken care of). If the generator's 'throw' method does not handle the exception, then it is re-raised in the outer block containing the 'for' statement. Second explanation: Reading the definition of the 'throw' method for generators in PEP 342, I naturally thought that the 'for' statement would abide by this new protocol. I was surprised to learn that it didn't. Since generators are nearly always used in a 'for' statement, how is this new method to be utilized? This isn't easily done. The code ends up looking like: g = foo(...) for input in g: while True: try: output = bar(input) break # from 'while', can't easily break from 'for' anymore... except Exception: input = g.throw(*sys.exc_info()) Yikes! > > > > To get __enter__ and __exit__ behavior for an iterator, can't you just > wrap it in class that provides that capability and calls close? Sure, contextlib.closing. But, just as it's nice that files support __enter__ and __exit__, it would be nice if other objects that need to be closed (sockets, generators, etc) did too. And, with the example set by 'file', one is lead to expect this support in these other cases... Since there is no need to clean up after iterators in general, but only for generators specifically; and since the BDFL has nixed my point #5, it makes sense to only add the __enter__ and __exit__ to generators. (And, by extension, itertools). > You might need itertools to have some support that extended iterator > class but that seems simpler. I don't follow you here. -bruce From greg.ewing at canterbury.ac.nz Sat Sep 6 04:20:38 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 06 Sep 2008 14:20:38 +1200 Subject: [Python-ideas] should there be a difference between generators and iterators? In-Reply-To: <48C12F32.5040405@gmail.com> References: <48BFF78C.3070102@gmail.com> <48C12F32.5040405@gmail.com> Message-ID: <48C1E8F6.1090507@canterbury.ac.nz> Bruce Frederiksen wrote: > what is happening > is that the 'finally' clause in a generator isn't being honored by 'for' > statements. It's not all that clear whether the for-loop should be doing anything special to force a generator to finalize if the loop is exited prematurely. The for-loop isn't necessarily the only thing using the iterator -- other code may want to carry on getting items from it, in which case you *don't* want it forcibly terminated. More generally, I'm a bit worried by all the extra complications that generators seem to be accruing. If it becomes a general expectation that an iterator based on other iterators is supposed to pass on all these special conditions, it's going to put a big burden on implementors of iterators, and turn what ought to be very simple and straightforward code into something convoluted. -- Greg From szport at gmail.com Mon Sep 8 10:47:27 2008 From: szport at gmail.com (Zaur Shibzoukhov) Date: Mon, 8 Sep 2008 12:47:27 +0400 Subject: [Python-ideas] should there be a difference between generators and iterators? Message-ID: Bruce Frederiksen wrote: >Since there is no need to clean up after iterators in general, but only >for generators specifically; and since the BDFL has nixed my point #5, >it makes sense to only add the __enter__ and __exit__ to generators. >(And, by extension, itertools). It makes more sense to allow extending of builtin types in order to support different protocols (not only protocol for "with" statement) From szport at gmail.com Tue Sep 9 17:16:08 2008 From: szport at gmail.com (Zaur Shibzoukhov) Date: Tue, 9 Sep 2008 19:16:08 +0400 Subject: [Python-ideas] About calling syntax Message-ID: Suppose I define a class: class Foo(object): # children = [] # def __init__(self, *args, **kw): if kw: self.__dict__.update(kw) if args: self.chidlren = list(args) In current syntax I have to write the folowing "initialization" code: foo = \ Foo( Foo( Foo(x=3,y=4), Foo(x=5,y=6), x=4, y=5 ), x=1, y=2 ) I can't write this code as follows (it seems more natural for me): foo = \ Foo( x=1, y=2, Foo( x=4, y=5, Foo(x=3,y=4), Foo(x=5,y=6)) ) ) Would be desirable to allow two equivalent forms of calling syntax in python: (, ) and (, ) ? Best regards, Zaur From leif.walsh at gmail.com Tue Sep 9 17:32:05 2008 From: leif.walsh at gmail.com (Leif Walsh) Date: Tue, 9 Sep 2008 11:32:05 -0400 Subject: [Python-ideas] About calling syntax In-Reply-To: References: Message-ID: On Tue, Sep 9, 2008 at 11:16 AM, Zaur Shibzoukhov wrote: > Would be desirable to allow two equivalent forms of calling syntax in python: > > (, ) > > and > > (, ) > > ? I think you are talking about PEP 3102: http://www.python.org/dev/peps/pep-3102/ -- Cheers, Leif From dangyogi at gmail.com Tue Sep 9 19:25:45 2008 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Tue, 09 Sep 2008 13:25:45 -0400 Subject: [Python-ideas] About calling syntax In-Reply-To: References: Message-ID: <48C6B199.3050700@gmail.com> Leif Walsh wrote: > On Tue, Sep 9, 2008 at 11:16 AM, Zaur Shibzoukhov wrote: > >> Would be desirable to allow two equivalent forms of calling syntax in python: >> >> (, ) >> >> and >> >> (, ) >> >> ? >> > > I think you are talking about PEP 3102: http://www.python.org/dev/peps/pep-3102/ > I don't think so. This PEP does not affect the syntax for calling a function. From g.brandl at gmx.net Tue Sep 9 20:23:22 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 09 Sep 2008 20:23:22 +0200 Subject: [Python-ideas] About calling syntax In-Reply-To: <48C6B199.3050700@gmail.com> References: <48C6B199.3050700@gmail.com> Message-ID: Bruce Frederiksen schrieb: > Leif Walsh wrote: >> On Tue, Sep 9, 2008 at 11:16 AM, Zaur Shibzoukhov wrote: >> >>> Would be desirable to allow two equivalent forms of calling syntax in python: >>> >>> (, ) >>> >>> and >>> >>> (, ) >>> >>> ? >>> >> >> I think you are talking about PEP 3102: http://www.python.org/dev/peps/pep-3102/ >> > I don't think so. This PEP does not affect the syntax for calling a > function. It does, in that you can do foo(*args, flag=True) which is currently a SyntaxError. However, exchanging poisitional and keyword arguments is not in the scope of the PEP, and isn't likely to have any future at all -- it's just too ambiguous when the called object has named arguments instead of a catch-all *args in its signature. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From talin at acm.org Wed Sep 10 06:19:52 2008 From: talin at acm.org (Talin) Date: Tue, 09 Sep 2008 21:19:52 -0700 Subject: [Python-ideas] About calling syntax In-Reply-To: References: <48C6B199.3050700@gmail.com> Message-ID: <48C74AE8.2020907@acm.org> Georg Brandl wrote: > Bruce Frederiksen schrieb: >> Leif Walsh wrote: >>> On Tue, Sep 9, 2008 at 11:16 AM, Zaur Shibzoukhov wrote: >>> >>>> Would be desirable to allow two equivalent forms of calling syntax in python: >>>> >>>> (, ) >>>> >>>> and >>>> >>>> (, ) >>>> >>>> ? >>>> >>> I think you are talking about PEP 3102: http://www.python.org/dev/peps/pep-3102/ >>> >> I don't think so. This PEP does not affect the syntax for calling a >> function. > > It does, in that you can do > > foo(*args, flag=True) > > which is currently a SyntaxError. > > However, exchanging poisitional and keyword arguments is not in the scope of > the PEP, and isn't likely to have any future at all -- it's just too ambiguous > when the called object has named arguments instead of a catch-all *args in > its signature. Correct. However, to address the OP's question, do this instead: foo = \ Foo(x=1, y=2, [Foo( x=4, y=5, [Foo(x=3,y=4), Foo(x=5,y=6))] )] ) Or, have Foo() return a callable: foo = \ Foo(x=1, y=2)( Foo(x=4, y=5)( Foo(x=3,y=4)(), Foo(x=5,y=6)()) ) ) Or any number of other variations... -- Talin > > Georg > From szport at gmail.com Wed Sep 10 08:22:43 2008 From: szport at gmail.com (Zaur Shibzoukhov) Date: Wed, 10 Sep 2008 10:22:43 +0400 Subject: [Python-ideas] About calling syntax Message-ID: Talin wrote: >However, to address the OP's question, do this instead: >foo = \ > Foo(x=1, y=2)( > Foo(x=4, y=5)( > Foo(x=3,y=4)(), > Foo(x=5,y=6)()) > ) > ) I use this form in my real coding. Georg Brandl wrote: > However, exchanging poisitional and keyword arguments is not in the scope of > the PEP, and isn't likely to have any future at all -- it's just too ambiguous > when the called object has named arguments instead of a catch-all *args in > its signature. More precisely I asked about the following thing. Let def foo(*args, **kw): ... So in call: foo(x=2,y=3, a,b,c) args = [a,b,c] and kw = {'x':2, 'y':3} in call: foo(a,b,c, x=2, y=3) args = [a,b,c] and kw = {'x':2, 'y':3} too. It's just simmetrical form of calling syntax. Best regards, Zaur From dangyogi at gmail.com Wed Sep 10 16:20:48 2008 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Wed, 10 Sep 2008 10:20:48 -0400 Subject: [Python-ideas] should there be a difference between generators and iterators? In-Reply-To: <48C1E8F6.1090507@canterbury.ac.nz> References: <48BFF78C.3070102@gmail.com> <48C12F32.5040405@gmail.com> <48C1E8F6.1090507@canterbury.ac.nz> Message-ID: <48C7D7C0.8020509@gmail.com> Greg Ewing wrote: > Bruce Frederiksen wrote: > >> what is happening is that the 'finally' clause in a generator isn't >> being honored by 'for' statements. > > It's not all that clear whether the for-loop should be > doing anything special to force a generator to finalize > if the loop is exited prematurely. The for-loop isn't > necessarily the only thing using the iterator -- other > code may want to carry on getting items from it, in > which case you *don't* want it forcibly terminated. > > More generally, I'm a bit worried by all the extra > complications that generators seem to be accruing. If > it becomes a general expectation that an iterator based > on other iterators is supposed to pass on all these > special conditions, it's going to put a big burden on > implementors of iterators, and turn what ought to be > very simple and straightforward code into something > convoluted. > It doesn't seem that difficult to pass 'close' on to your child iterator. Passing 'throw' on might require interpreting non-error results from the child iterator, which would depend on what your iterator is doing. If there are places where this is too complicated, then don't implement 'throw' for that iterator. And the same thing for 'send' as for 'throw'. But the most important one is 'close', because this runs the 'finally' clause in the generator. Looking through the itertools, a user of any of these except 'chain' can capture the input iterable(s) in a with closing clause prior to calling the itertools function. So for everything except the 'chain' function the 'close' is more of a "nice to have" rather than a "must have". But for the 'chain' function, the caller of 'chain' does not have access to the iterables generated by the iterable passed to 'chain' (specifically with 'from_iterable'). Thus, adding 'close' to 'chain' is more of a "must have". (Well, I guess you could argue that the user should write his own chain function with a 'close' capability rather than using the itertools function). So 'chain' would become equivalent to: def chain(*iterables): # chain('ABC', 'DEF') --> A B C D E F it = None try: for it in iterables: # <---- 'it' is the inaccessible iterator! for element in it: yield element finally: if hasattr(it, 'close'): it.close() if hasattr(iterables, 'close'): iterables.close() -bruce From dangyogi at gmail.com Wed Sep 10 16:27:22 2008 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Wed, 10 Sep 2008 10:27:22 -0400 Subject: [Python-ideas] should there be a difference between generators and iterators? In-Reply-To: References: Message-ID: <48C7D94A.5060803@gmail.com> Zaur Shibzoukhov wrote: > Bruce Frederiksen wrote: > >> Since there is no need to clean up after iterators in general, but only >> for generators specifically; and since the BDFL has nixed my point #5, >> it makes sense to only add the __enter__ and __exit__ to generators. >> (And, by extension, itertools). >> > > It makes more sense to allow extending of builtin types in order to > support different protocols > (not only protocol for "with" statement) > Do you mean the python programmer should do: import types types.GeneratorType.__enter__ = lambda self: self types.GeneratorType.__exit__ = lambda self: self.close() So that the objects returned by his generator functions (functions containing 'yield') have context manager capability? From bruce at leapyear.org Wed Sep 10 17:24:12 2008 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 10 Sep 2008 08:24:12 -0700 Subject: [Python-ideas] About calling syntax In-Reply-To: References: Message-ID: On Tue, Sep 9, 2008 at 11:22 PM, Zaur Shibzoukhov wrote: > in call: foo(x=2,y=3, a,b,c) > args = [a,b,c] and kw = {'x':2, 'y':3} > > in call: foo(a,b,c, x=2, y=3) > args = [a,b,c] and kw = {'x':2, 'y':3} too. > > It's just simmetrical form of calling syntax. > But why should you be forced to separate them like that. Why not foo(a,x=2,b,y=3,c,z=4) args = [a,b,c] and kw = {'x':2, 'y':3, 'z':4} I'm sure there are scenarios where this is useful but I think in general it would be less readable. As it is, I can't miss a positional argument in the middle of a long list of keyword arguments: foo(a=b,c=d,e=f,g=h,i=j,k=l,m) --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From szport at gmail.com Wed Sep 10 20:49:08 2008 From: szport at gmail.com (Zaur Shibzoukhov) Date: Wed, 10 Sep 2008 22:49:08 +0400 Subject: [Python-ideas] Fwd: should there be a difference between generators and iterators? In-Reply-To: References: <48C7D94A.5060803@gmail.com> Message-ID: ---------- Forwarded message ---------- From: Zaur Shibzoukhov Date: 2008/9/10 Subject: Re: [Python-ideas] should there be a difference between generators and iterators? To: Bruce Frederiksen I think it should be as import types types.GeneratorType.__enter__ = MethodType(lambda self: self, types.GeneratorType) types.GeneratorType.__exit__ = MethodType(lambda self: self.close(), types.GeneratorType) >So that the objects returned by his generator functions (functions containing 'yield') >have context manager capability? IMHO yes. Zaur 2008/9/10 Bruce Frederiksen : > Zaur Shibzoukhov wrote: >> >> Bruce Frederiksen wrote: >> >>> >>> Since there is no need to clean up after iterators in general, but only >>> for generators specifically; and since the BDFL has nixed my point #5, >>> it makes sense to only add the __enter__ and __exit__ to generators. >>> (And, by extension, itertools). >>> >> >> It makes more sense to allow extending of builtin types in order to >> support different protocols >> (not only protocol for "with" statement) >> > > Do you mean the python programmer should do: > > import types > > types.GeneratorType.__enter__ = lambda self: self > types.GeneratorType.__exit__ = lambda self: self.close() > > So that the objects returned by his generator functions (functions > containing 'yield') have context manager capability? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cliff at develix.com Wed Sep 10 20:43:00 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 10 Sep 2008 11:43:00 -0700 Subject: [Python-ideas] Statements vs Expressions... why? Message-ID: <1221072180.12242.20.camel@portable-evil> Greetings, Something that has started to annoy me in the last couple of years is the fact that most Python control statements cannot be used as expressions. I feel this is a pretty deep limitation and personally I don't feel it's well-justified. As I understand it, the reason for the distinction mostly has to do with the premise "flat is better than nested", which I can understand, but I don't think carries enough weight anymore. Specifically, I'd like to see things like "if" statements, "for" loops, etc, become expressions. This would not preclude them from being used as if they were statements (an expression can stand alone on a line of Python code), but would add a lot of expressiveness to the language as well as make Python more viable for creating DSLs. Additionally, removing statements from Python would also allow the language to be simplified. No need for a ternary "if" operator with different semantics than a normal "if" statement, "for" loops would be brought closer to generators in functionality, and the perceived limitations of lambda would disappear, amongst other things. We'd gain a lot of features found in languages like Lisp and Ruby as a side-effect (i.e. anonymous code blocks). Overall it seems this design decision is specifically geared toward forcing programmers into an imperative style in order to enforce program readability. In Python 1.5, this made a bit of sense, but as Python has "matured" (or in my view, gotten over-complicated) this makes much less sense. Many parts of Python's extensive syntax are explicit workarounds to this design decision. So on the one hand we have the perceived danger that programmers will write nested code and on the other we have an ever-expanding syntax. I'd take the former any day. I've not delved into the internals of the Python interpreter to check, but I suspect that converting most statements to expressions would not be very difficult (changing the grammar definition and generated bytecode a small amount in most cases). Any thoughts on this? I'm sure it's been brought up before, but I haven't found any definitive discussions on why this rather arbitrary design decision continues to hold in the face of a general migration away from imperative languages (especially when it seems it could be changed without much backwards-compatibility issues). Regards, Cliff From szport at gmail.com Wed Sep 10 20:50:05 2008 From: szport at gmail.com (Zaur Shibzoukhov) Date: Wed, 10 Sep 2008 22:50:05 +0400 Subject: [Python-ideas] Fwd: About calling syntax In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: Zaur Shibzoukhov Date: 2008/9/10 Subject: Re: [Python-ideas] About calling syntax To: Bruce Leban 2008/9/10 Bruce Leban > I'm sure there are scenarios where this is useful but I think in general it > would be less readable. As it is, I can't miss a positional argument in the > middle of a long list of keyword arguments: > > foo(a=b,c=d,e=f,g=h,i=j,k=l,m) > > But programmer could select most readable form of function call (as the case may be). Best regards, Zaur -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at cheimes.de Wed Sep 10 21:46:48 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 10 Sep 2008 21:46:48 +0200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221072180.12242.20.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> Message-ID: Cliff Wells wrote: > Any thoughts on this? I'm sure it's been brought up before, but I > haven't found any definitive discussions on why this rather arbitrary > design decision continues to hold in the face of a general migration > away from imperative languages (especially when it seems it could be > changed without much backwards-compatibility issues). Two thoughts: Please elaborate how you like to change the syntax of Python. I like to see some concrete examples how your syntax would look like. I also like to know how your are planing to implement features like lazy evaluation. The if else ternary operator statement is evaluated lazy. The same construct as expression wouldn't be lazy any more. Secondly any syntax change won't happen until we start planing Python 4000 ;) Christian From bwinton at latte.ca Wed Sep 10 21:47:19 2008 From: bwinton at latte.ca (Blake Winton) Date: Wed, 10 Sep 2008 15:47:19 -0400 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221072180.12242.20.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> Message-ID: <48C82447.60700@latte.ca> Cliff Wells wrote: > I'd like to see things like "if" statements, "for" loops, > etc, become expressions. [...] > Any thoughts on this? I'm sure it's been brought up before, but I > haven't found any definitive discussions on why this rather arbitrary > design decision continues to hold in the face of a general migration > away from imperative languages (especially when it seems it could be > changed without much backwards-compatibility issues). Can you show me what an "if" in a "lambda" used in a function call would look like? My major complaint with the statements-as-expressions is that multi-line statements really don't look good when used in expression contexts. Perhaps you have a good suggestion for the syntax, and if so, I'ld love to see it. > I've not delved into the internals of the Python interpreter to check, > but I suspect that converting most statements to expressions would not > be very difficult (changing the grammar definition and generated > bytecode a small amount in most cases). Well, I suspect that delving into the internals to check this theory would make people take your proposal a lot more seriously. Heck, I'ld go so far as to say that without doing that, it's all just talk, and talk is cheap. ;) Thanks, Blake. From arnodel at googlemail.com Wed Sep 10 21:55:00 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 10 Sep 2008 20:55:00 +0100 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221072180.12242.20.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> Message-ID: [Sorry for the private reply earlier] On 10 Sep 2008, at 19:43, Cliff Wells wrote: > Greetings, > > Something that has started to annoy me in the last couple of years is > the fact that most Python control statements cannot be used as > expressions. I feel this is a pretty deep limitation and personally I > don't feel it's well-justified. > > As I understand it, the reason for the distinction mostly has to do > with > the premise "flat is better than nested", which I can understand, > but I > don't think carries enough weight anymore. > > Specifically, I'd like to see things like "if" statements, "for" > loops, > etc, become expressions. This would not preclude them from being used > as if they were statements (an expression can stand alone on a line of > Python code), but would add a lot of expressiveness to the language as > well as make Python more viable for creating DSLs. > Can you post some sample code to illustrate how statements could be used as expressions? Do you propose that we write: y = if x == 0: 0 else: exp(x**-2) instead of: y = 0 if x == 0 else exp(x**-2) ? Or, how would you write factors = [x for x in range(2, n) if n % x == 0] ? Something like this maybe: factors = for x in range(2, n): if n % x == 0: x Or do you suggest something else? > Additionally, removing statements from Python would also allow the > language to be simplified. No need for a ternary "if" operator with > different semantics than a normal "if" statement, "for" loops would be > brought closer to generators in functionality, and the perceived > limitations of lambda would disappear, amongst other things. We'd > gain > a lot of features found in languages like Lisp and Ruby as a side- > effect > (i.e. anonymous code blocks). > > Overall it seems this design decision is specifically geared toward > forcing programmers into an imperative style in order to enforce > program > readability. In Python 1.5, this made a bit of sense, but as Python > has > "matured" (or in my view, gotten over-complicated) this makes much > less > sense. Many parts of Python's extensive syntax are explicit > workarounds > to this design decision. So on the one hand we have the perceived > danger that programmers will write nested code and on the other we > have > an ever-expanding syntax. I'd take the former any day. > So do you think readability is not as important now as it was? > I've not delved into the internals of the Python interpreter to check, > but I suspect that converting most statements to expressions would not > be very difficult (changing the grammar definition and generated > bytecode a small amount in most cases). > > Any thoughts on this? I'm sure it's been brought up before, but I > haven't found any definitive discussions on why this rather arbitrary > design decision continues to hold in the face of a general migration > away from imperative languages (especially when it seems it could be > changed without much backwards-compatibility issues). I think to call this feature of Python an arbitrary design decision is a misjudgement. To me it is central to the identity of Python. -- Arnaud From bwinton at latte.ca Wed Sep 10 21:51:09 2008 From: bwinton at latte.ca (Blake Winton) Date: Wed, 10 Sep 2008 15:51:09 -0400 Subject: [Python-ideas] Fwd: About calling syntax In-Reply-To: References: Message-ID: <48C8252D.8070004@latte.ca> Zaur Shibzoukhov wrote: > 2008/9/10 Bruce Leban > > > I'm sure there are scenarios where this is useful but I think in > general it would be less readable. As it is, I can't miss a > positional argument in the middle of a long list of keyword arguments: > > foo(a=b,c=d,e=f,g=h,i=j,k=l,m) > > > But programmer could select most readable form of function call (as the > case may be). But some of them wouldn't, and I would have to learn all the forms of function call to read other people's code. Additional forms are easy for the writer, but hard for the reader, and I read far more code than I write. To say it a different way, programmers _could_ write easy-to-understand Perl, but they can also write hard-to-understand Perl, and in my experience when presented with that choice, most of them choose the hard-to-understand route. ;) Later, Blake. From cliff at develix.com Wed Sep 10 23:18:36 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 10 Sep 2008 14:18:36 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> Message-ID: <1221081516.12242.53.camel@portable-evil> On Wed, 2008-09-10 at 20:55 +0100, Arnaud Delobelle wrote: > [Sorry for the private reply earlier] > > On 10 Sep 2008, at 19:43, Cliff Wells wrote: > > > Greetings, > > > > Something that has started to annoy me in the last couple of years is > > the fact that most Python control statements cannot be used as > > expressions. I feel this is a pretty deep limitation and personally I > > don't feel it's well-justified. > > > > As I understand it, the reason for the distinction mostly has to do > > with > > the premise "flat is better than nested", which I can understand, > > but I > > don't think carries enough weight anymore. > > > > Specifically, I'd like to see things like "if" statements, "for" > > loops, > > etc, become expressions. This would not preclude them from being used > > as if they were statements (an expression can stand alone on a line of > > Python code), but would add a lot of expressiveness to the language as > > well as make Python more viable for creating DSLs. > > > > Can you post some sample code to illustrate how statements could be > used as expressions? > > Do you propose that we write: > > y = if x == 0: > 0 > else: > exp(x**-2) > > instead of: > > y = 0 if x == 0 else exp(x**-2) Yes, and parentheses could be used to disambiguate, as anywhere else. > ? Or, how would you write > > factors = [x for x in range(2, n) if n % x == 0] > > ? Something like this maybe: > > factors = for x in range(2, n): > if n % x == 0: > x > > Or do you suggest something else? These are correct, albeit simple examples (you are using only assignment examples). To give something more interesting, I'd like to be able to do things like this: dispatch = { '1': lambda x: ( for i in range(x): if not x % 2: yield 0 else: yield 1 ), '2': lambda x: ( for i in range(x): yield i ) } for i in dispatch[val](1): print i Note that this isn't just about lambda, but rather general expressiveness. To clarify, I propose *not* making syntax changes that add to the language or alter the meaning of existing code. Rather my proposal is along the lines of "relax this particular syntax requirement" and let the chips fall where they may. In other words, Python 2.x code would still work, it simply would not take advantage of a particular coding style now available to it. The key difference between a statement and an expression is that 1) an expression has a return value, a statement does not 2) statements cannot be used inside of expressions In short, statements are more-or-less castrated expressions. They have no other special features. > > Additionally, removing statements from Python would also allow the > > language to be simplified. No need for a ternary "if" operator with > > different semantics than a normal "if" statement, "for" loops would be > > brought closer to generators in functionality, and the perceived > > limitations of lambda would disappear, amongst other things. We'd > > gain > > a lot of features found in languages like Lisp and Ruby as a side- > > effect > > (i.e. anonymous code blocks). > > > > Overall it seems this design decision is specifically geared toward > > forcing programmers into an imperative style in order to enforce > > program > > readability. In Python 1.5, this made a bit of sense, but as Python > > has > > "matured" (or in my view, gotten over-complicated) this makes much > > less > > sense. Many parts of Python's extensive syntax are explicit > > workarounds > > to this design decision. So on the one hand we have the perceived > > danger that programmers will write nested code and on the other we > > have > > an ever-expanding syntax. I'd take the former any day. > > > > So do you think readability is not as important now as it was? Of course not. I'm saying that due to this limitation, readability is going down the drain even faster. It's getting to the point where it's not possible to keep the whole of Python in your head and I feel that a significant portion of this is due to this particular inflexibility. Further, I feel that this limitation forces programmers into using hacks and magic or overly spread-out code, which itself leads to readability concerns. Having used Python for around a decade, I'm quite aware of the fact that you can happily write tons and tons of nice code with Python in its current state. However, because of the direction of my work (developing a internal DSL in Python) I've suddenly become aware of this glass ceiling. I'd bumped into it before back when I was doing a lot of GUI development, but blamed it on lambda rather than realizing that it wasn't lambda so much as what I am bringing up now. > > I've not delved into the internals of the Python interpreter to check, > > but I suspect that converting most statements to expressions would not > > be very difficult (changing the grammar definition and generated > > bytecode a small amount in most cases). > > > > Any thoughts on this? I'm sure it's been brought up before, but I > > haven't found any definitive discussions on why this rather arbitrary > > design decision continues to hold in the face of a general migration > > away from imperative languages (especially when it seems it could be > > changed without much backwards-compatibility issues). > > I think to call this feature of Python an arbitrary design decision is > a misjudgement. To me it is central to the identity of Python. Sorry, I misspoke: it's not an arbitrary *decision*, but it is an arbitrary *distinction*. As I said, I'm aware of why this decision was originally made (prevention of nested code, a.k.a. fear of lisp), but the distinction itself is completely artificial, created to enforce a model of programming, rather than for technical or performance reasons. I agree that it is a distinguishing feature of Python, but I don't think it is central to Python's identity. That is, it wouldn't be "not Python" were it removed. Regards, Cliff From cliff at develix.com Wed Sep 10 23:22:43 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 10 Sep 2008 14:22:43 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> Message-ID: <1221081763.12242.58.camel@portable-evil> On Wed, 2008-09-10 at 21:46 +0200, Christian Heimes wrote: > Cliff Wells wrote: > > Any thoughts on this? I'm sure it's been brought up before, but I > > haven't found any definitive discussions on why this rather arbitrary > > design decision continues to hold in the face of a general migration > > away from imperative languages (especially when it seems it could be > > changed without much backwards-compatibility issues). > > Two thoughts: > > Please elaborate how you like to change the syntax of Python. No changes. Simply lifting of a particular restriction. > I like to > see some concrete examples how your syntax would look like. I also like > to know how your are planing to implement features like lazy evaluation. > The if else ternary operator statement is evaluated lazy. The same > construct as expression wouldn't be lazy any more. Why not? a = ( if a > 1 then: long_calculation() else: other_long_calculation() ) Clearly only one of these blocks would be evaluated at runtime. > > Secondly any syntax change won't happen until we start planing Python > 4000 ;) Yes, that's my expectation, although hopefully PyPy will make some of these things possible to experiment with well before then =) Cliff From santagada at gmail.com Wed Sep 10 23:52:45 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 10 Sep 2008 18:52:45 -0300 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221081763.12242.58.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> Message-ID: <7E2ABF90-71A8-4BD6-9458-71AE6C5E9937@gmail.com> On Sep 10, 2008, at 6:22 PM, Cliff Wells wrote: > On Wed, 2008-09-10 at 21:46 +0200, Christian Heimes wrote: >> Cliff Wells wrote: >>> Any thoughts on this? I'm sure it's been brought up before, but I >>> haven't found any definitive discussions on why this rather >>> arbitrary >>> design decision continues to hold in the face of a general migration >>> away from imperative languages (especially when it seems it could be >>> changed without much backwards-compatibility issues). >> >> Two thoughts: >> >> Please elaborate how you like to change the syntax of Python. > > No changes. Simply lifting of a particular restriction. Two restrictions, both that statements can be used in place of expressions and that statements now return values. But please explain how to do it in a way that is clear. It's a plus if it is backwards compatible :) >> I like to >> see some concrete examples how your syntax would look like. I also >> like >> to know how your are planing to implement features like lazy >> evaluation. >> The if else ternary operator statement is evaluated lazy. The same >> construct as expression wouldn't be lazy any more. > > Why not? > > a = ( > if a > 1 then: > long_calculation() > else: > other_long_calculation() > ) > > Clearly only one of these blocks would be evaluated at runtime. And now ifs return a value >> >> Secondly any syntax change won't happen until we start planing Python >> 4000 ;) > > Yes, that's my expectation, although hopefully PyPy will make some of > these things possible to experiment with well before then =) You can do that now in PyPy or any other python version... or maybe use Logix to show us a proof of concept version: http://www.livelogix.net/logix/ Now if I remeber correctly this is already the case of logix, look at: http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html#3.1 Why would you need lambda then? couldn't you just write x = def (x,y): return x+y ? From cliff at develix.com Wed Sep 10 23:54:44 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 10 Sep 2008 14:54:44 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221072180.12242.20.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> Message-ID: <1221083684.12242.75.camel@portable-evil> On Wed, 2008-09-10 at 11:43 -0700, Cliff Wells wrote: > Specifically, I'd like to see things like "if" statements, "for" loops, > etc, become expressions. This would not preclude them from being used > as if they were statements (an expression can stand alone on a line of > Python code), but would add a lot of expressiveness to the language as > well as make Python more viable for creating DSLs. Because I know that the question "what's your use case" will inevitably be forwarded, I will explain the exact use case that has driven me to distraction over this limitation. I developed and maintain a template engine written in Python: http://breve.twisty-industries.com/ As you can see from the example on the main page, Breve is written as an internal DSL (that is, it compiles directly to Python bytecode... Breve templates *are* Python). This means that any limitation in Python is inherently a limitation in Breve. There were several design decisions to be made when designing Breve, the first and foremost being "will Breve templates be full-blown Python programs or will they be limited to expressions?". I chose the latter for a few reasons: 1) Python expressions map more directly to the generated XML/HTML output. In other words, a Breve template has the same structure (if not syntax) as the desired result. 2) It would have required additional syntax within the template to allow for all of Python to be supported. Overall, I'm happy with my choice, but it forces some mental twister when trying to do particular operations in Breve, notably presentation-logic and looping. A loop must be presented as a list comprehension. "If" logic must be done using either 2.5's ternary if operator or using short-circuit logical operations, i.e. "test(value) and (this)". At one point I actually implemented custom tags for doing logical operations, but due to the inability to provide short-circuiting, they were less than satisfactory. This isn't fatal to Breve (albeit annoying), but it made me painfully aware of this self-imposed limitation in Python. It also made me wonder what other concepts this limitation has prevented me from grokking over this last decade. It's generally accepted that a language constrains not only the solutions to a problem, but also the *possible* solutions that a programmer will see. This is as much of a concern to me as any practical concerns today. Regards, Cliff From rhamph at gmail.com Wed Sep 10 23:57:46 2008 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 10 Sep 2008 15:57:46 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221081516.12242.53.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> Message-ID: On Wed, Sep 10, 2008 at 3:18 PM, Cliff Wells wrote: > Further, I feel that this limitation forces programmers into using hacks > and magic or overly spread-out code, which itself leads to readability > concerns. Having used Python for around a decade, I'm quite aware of > the fact that you can happily write tons and tons of nice code with > Python in its current state. However, because of the direction of my > work (developing a internal DSL in Python) I've suddenly become aware of > this glass ceiling. I'd bumped into it before back when I was doing a > lot of GUI development, but blamed it on lambda rather than realizing > that it wasn't lambda so much as what I am bringing up now. Python is not intended for DSLs. Really, don't do it. Python is for python code. If you want another language, write your own parser. I hear lisp is simple to parse, and has no annoying statements to hold you back! Seriously though, there is an advantage to basing so much on statements rather than expressions. We're specialized for one statement per line, which is the most common case, and it allowed us to have extraneous semicolons, braces, or whatever. Readability benefits, style consistency benefits. Now there are some use cases that suffer here, including the one you just gave: defining a dispatch dict with the functions inline. The best I could do is define the dict first, then stick a decorator on each function to register them. That's still ugly though. A creative solution is needed, but none come to mind. An example where this has happened before is the with-statement, which is spectacularly successful IMO. Now, you may notice it could have been done in a library rather than core syntax if generic anonymous blocks were allowed ? so what? The library is still part of the language! It's still something that has to be learned. And the syntax would be substantially uglier using a generic mechanism, rather than the specialized with-statement syntax. -- Adam Olsen, aka Rhamphoryncus From cliff at develix.com Thu Sep 11 00:13:56 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 10 Sep 2008 15:13:56 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <7E2ABF90-71A8-4BD6-9458-71AE6C5E9937@gmail.com> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <7E2ABF90-71A8-4BD6-9458-71AE6C5E9937@gmail.com> Message-ID: <1221084836.12242.89.camel@portable-evil> On Wed, 2008-09-10 at 18:52 -0300, Leonardo Santagada wrote: > On Sep 10, 2008, at 6:22 PM, Cliff Wells wrote: > > > On Wed, 2008-09-10 at 21:46 +0200, Christian Heimes wrote: > >> Cliff Wells wrote: > >>> Any thoughts on this? I'm sure it's been brought up before, but I > >>> haven't found any definitive discussions on why this rather > >>> arbitrary > >>> design decision continues to hold in the face of a general migration > >>> away from imperative languages (especially when it seems it could be > >>> changed without much backwards-compatibility issues). > >> > >> Two thoughts: > >> > >> Please elaborate how you like to change the syntax of Python. > > > > No changes. Simply lifting of a particular restriction. > > Two restrictions, both that statements can be used in place of > expressions and that statements now return values. But please explain > how to do it in a way that is clear. It's a plus if it is backwards > compatible :) Well, one or two, depending on how you word it ;-) My position is to simply do away with statements and provide equivalent expressions. As far as backwards-compatibility, this is legal Python: 1 2 3 It doesn't do anything, but it is valid syntax. This demonstrates that expressions can be valid when used independent of a statement, therefore, the following is also legal (assuming "if" were now implemented as an expression): if (a==1): pass > >> I like to > >> see some concrete examples how your syntax would look like. I also > >> like > >> to know how your are planing to implement features like lazy > >> evaluation. > >> The if else ternary operator statement is evaluated lazy. The same > >> construct as expression wouldn't be lazy any more. > > > > Why not? > > > > a = ( > > if a > 1 then: > > long_calculation() > > else: > > other_long_calculation() > > ) > > > > Clearly only one of these blocks would be evaluated at runtime. > > And now ifs return a value Which you are not required to use, just as today: def foo(x): return x**2 foo(2) > > >> > >> Secondly any syntax change won't happen until we start planing Python > >> 4000 ;) > > > > Yes, that's my expectation, although hopefully PyPy will make some of > > these things possible to experiment with well before then =) > > You can do that now in PyPy or any other python version... or maybe > use Logix to show us a proof of concept version: http://www.livelogix.net/logix/ > > Now if I remeber correctly this is already the case of logix, look at: > http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html#3.1 Yes, Logix is quite interesting. Unfortunately the author has discontinued work on it (and was, in fact, unreachable when I last tried to contact him). He also mentioned retargeting to another VM (rather than the Python VM) which made it less appealing to me. It is interesting that you mention Logix, since it demonstrates that it's quite possible to be backward-compatible with this particular change. > Why would you need lambda then? couldn't you just write x = def (x,y): > return x+y ? Yes, although I suspect that this particular change to "def" would be more substantial and "lambda" is currently equivalent. Although this might satisfy GvR's inclination to dump lambda at some future point ;-) Cliff From cliff at develix.com Thu Sep 11 00:39:49 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 10 Sep 2008 15:39:49 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> Message-ID: <1221086389.12242.116.camel@portable-evil> On Wed, 2008-09-10 at 15:57 -0600, Adam Olsen wrote: > On Wed, Sep 10, 2008 at 3:18 PM, Cliff Wells wrote: > > Further, I feel that this limitation forces programmers into using hacks > > and magic or overly spread-out code, which itself leads to readability > > concerns. Having used Python for around a decade, I'm quite aware of > > the fact that you can happily write tons and tons of nice code with > > Python in its current state. However, because of the direction of my > > work (developing a internal DSL in Python) I've suddenly become aware of > > this glass ceiling. I'd bumped into it before back when I was doing a > > lot of GUI development, but blamed it on lambda rather than realizing > > that it wasn't lambda so much as what I am bringing up now. > > Python is not intended for DSLs. Really, don't do it. Python is for > python code. The DSL I work on *is* Python code. And really, this is the first time I've heard anyone assert anything like this. Python is a general-purpose language. It's not VBA ;-) DSL's are an extremely useful concept. To summarily dispatch the whole of them with such an assertion is pretty much bolstering my argument: you've just asserted that Python is inherently limited in scope. > If you want another language, write your own parser. I > hear lisp is simple to parse, and has no annoying statements to hold > you back! Ah, except Python is the language I like in every way, *except* for this one particular wart. Really, had I not entered new programming domains and tried to take Python with me, I'd probably never have had a complaint. Also, external parsers defeat the entire reasoning behind internal DSL's (taking advantage of an established VM/compiler, requiring users to learn a new syntax in addition to their primary programming language). > Seriously though, there is an advantage to basing so much on > statements rather than expressions. We're specialized for one > statement per line, which is the most common case, Clearly it's the most common case in existing Python code since nothing else is allowed. But frankly, even Javascript doesn't follow this idiom anymore. Expression-oriented languages have seen a rebirth for good reasons (although I admit I'm none-to-fond of many of them, for various reasons). > and it allowed us > to have extraneous semicolons, braces, or whatever. Not following this. You mean to *not* have extraneous syntax? > Readability benefits, style consistency benefits. I strongly disagree. The artificial distinction between statements and expressions is the definition of *inconsistent*. Why do we have two versions of the "if" conditional? Why do we have "for" loops *and* list comprehensions? They express the same ideas, but the limitations in one required growing the language another direction. In short, we could have had a single, more powerful construct in place of two lesser constructs and simultaneously had less syntax to memorize and more consistency across the board. > Now there are some use cases that suffer here, including the one you > just gave: defining a dispatch dict with the functions inline. The > best I could do is define the dict first, then stick a decorator on > each function to register them. That's still ugly though. A creative > solution is needed, but none come to mind. That's because there is none. And this is my fundamental problem: it's not so much that it's hard to do in Python, it's that you *cannot* do it in Python. No amount of creativity, time, or experience will help, and this is disappointing. I won't pretend that any example we might toss up in here won't appear contrived, but there are definite cases where readability can be substantially enhanced with such structures. > An example where this has happened before is the with-statement, which > is spectacularly successful IMO. Now, you may notice it could have > been done in a library rather than core syntax if generic anonymous > blocks were allowed ? so what? The library is still part of the > language! It's still something that has to be learned. And the > syntax would be substantially uglier using a generic mechanism, rather > than the specialized with-statement syntax. The "so what" is that it could *only* be implemented by the core devs. It was not possible for an average (or even above-average) Python programmer to write such a library, whereas it *could* have been had the language not prohibited it. Regards, Cliff From rhamph at gmail.com Thu Sep 11 01:16:16 2008 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 10 Sep 2008 17:16:16 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221086389.12242.116.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> Message-ID: On Wed, Sep 10, 2008 at 4:39 PM, Cliff Wells wrote: > On Wed, 2008-09-10 at 15:57 -0600, Adam Olsen wrote: >> On Wed, Sep 10, 2008 at 3:18 PM, Cliff Wells wrote: >> > Further, I feel that this limitation forces programmers into using hacks >> > and magic or overly spread-out code, which itself leads to readability >> > concerns. Having used Python for around a decade, I'm quite aware of >> > the fact that you can happily write tons and tons of nice code with >> > Python in its current state. However, because of the direction of my >> > work (developing a internal DSL in Python) I've suddenly become aware of >> > this glass ceiling. I'd bumped into it before back when I was doing a >> > lot of GUI development, but blamed it on lambda rather than realizing >> > that it wasn't lambda so much as what I am bringing up now. >> >> Python is not intended for DSLs. Really, don't do it. Python is for >> python code. > > The DSL I work on *is* Python code. And really, this is the first time > I've heard anyone assert anything like this. Python is a > general-purpose language. It's not VBA ;-) > > DSL's are an extremely useful concept. To summarily dispatch the whole > of them with such an assertion is pretty much bolstering my argument: > you've just asserted that Python is inherently limited in scope. I agree, DSL's are useful. They're just not something python currently supports well. They're the use-case you need to justify against the substantial changes you propose. Maybe it's worth it, or maybe it's better to add an indent-sensitive string literal that would then allow *arbitrary* DSL syntax? >> If you want another language, write your own parser. I >> hear lisp is simple to parse, and has no annoying statements to hold >> you back! > > Ah, except Python is the language I like in every way, *except* for this > one particular wart. Really, had I not entered new programming domains > and tried to take Python with me, I'd probably never have had a > complaint. > > Also, external parsers defeat the entire reasoning behind internal DSL's > (taking advantage of an established VM/compiler, requiring users to > learn a new syntax in addition to their primary programming language). I appreciate what you're saying here, and feel much the same way about my own pet-features, but this is a really poor argument. Everybody has just one wafer-thin feature they'd like to add. >> Seriously though, there is an advantage to basing so much on >> statements rather than expressions. We're specialized for one >> statement per line, which is the most common case, > > Clearly it's the most common case in existing Python code since nothing > else is allowed. But frankly, even Javascript doesn't follow this > idiom anymore. Expression-oriented languages have seen a rebirth for > good reasons (although I admit I'm none-to-fond of many of them, for > various reasons). Again, "XYZ language has it" is an ineffective argument. >> and it allowed us >> to have extraneous semicolons, braces, or whatever. > > Not following this. You mean to *not* have extraneous syntax? Consider a loop in C vs in python for (i = 0; i < 50; i++) { a(); b(); } for i in range(50): a() b() We avoid the semicolons and braces because the newline and indentation indicate the same thing. That's what a statement it. Perhaps you can retain this with your proposal, but only because a statement is still the default, with expression as a rarely used option. >> Readability benefits, style consistency benefits. > > I strongly disagree. The artificial distinction between statements and > expressions is the definition of *inconsistent*. Why do we have two > versions of the "if" conditional? Why do we have "for" loops *and* list > comprehensions? They express the same ideas, but the limitations in one > required growing the language another direction. In short, we could > have had a single, more powerful construct in place of two lesser > constructs and simultaneously had less syntax to memorize and more > consistency across the board. I was referring to consistency of the programs, not the language itself. No silly arguments about brace position because they *are no braces*. There's actually a problem with trying to merge a for-loop and a list comprehension. A generator expression is the canonical generic form, but [genexp] would create a list containing a single genexp object. Likewise, a for-loop would become lazy, so without some extra syntax to evaluate it (and trigger the side effects!), your programs would cease to do anything. So you see, despite significant and obvious similarity between the features, there's some important differences as well. These are so obvious from the contexts that you don't even think of them, so clearly the mental load of having 3 (soon to be 5) different forms of looping is not all that great. Mental load is what really counts, not some abstract concept of complexity. >> Now there are some use cases that suffer here, including the one you >> just gave: defining a dispatch dict with the functions inline. The >> best I could do is define the dict first, then stick a decorator on >> each function to register them. That's still ugly though. A creative >> solution is needed, but none come to mind. > > That's because there is none. And this is my fundamental problem: it's > not so much that it's hard to do in Python, it's that you *cannot* do it > in Python. No amount of creativity, time, or experience will help, and > this is disappointing. I didn't mean to do it in Python. I meant to modify the language. > I won't pretend that any example we might toss up in here won't appear > contrived, but there are definite cases where readability can be > substantially enhanced with such structures. > >> An example where this has happened before is the with-statement, which >> is spectacularly successful IMO. Now, you may notice it could have >> been done in a library rather than core syntax if generic anonymous >> blocks were allowed ? so what? The library is still part of the >> language! It's still something that has to be learned. And the >> syntax would be substantially uglier using a generic mechanism, rather >> than the specialized with-statement syntax. > > The "so what" is that it could *only* be implemented by the core devs. > It was not possible for an average (or even above-average) Python > programmer to write such a library, whereas it *could* have been had the > language not prohibited it. As important as it is to extend the language via a library, somewhere you need to draw the line and start modifying the syntax or other fundamental builtins. The is a universal tenant, applying even to lisp (which has very little syntax, rather than extensible syntax). This is why we have a+b, rather than add(a, b). More syntax when it's worth it. So it's not that we don't want to allow extensibility - quite the opposite. It's that we want the common statement to be simpler, and the extra syntax hasn't been justified for your use cases. -- Adam Olsen, aka Rhamphoryncus From josiah.carlson at gmail.com Thu Sep 11 01:24:31 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Wed, 10 Sep 2008 16:24:31 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> Message-ID: On Wed, Sep 10, 2008 at 2:57 PM, Adam Olsen wrote: > On Wed, Sep 10, 2008 at 3:18 PM, Cliff Wells wrote: >> Further, I feel that this limitation forces programmers into using hacks >> and magic or overly spread-out code, which itself leads to readability >> concerns. Having used Python for around a decade, I'm quite aware of >> the fact that you can happily write tons and tons of nice code with >> Python in its current state. However, because of the direction of my >> work (developing a internal DSL in Python) I've suddenly become aware of >> this glass ceiling. I'd bumped into it before back when I was doing a >> lot of GUI development, but blamed it on lambda rather than realizing >> that it wasn't lambda so much as what I am bringing up now. > > Python is not intended for DSLs. Really, don't do it. Python is for > python code. If you want another language, write your own parser. I > hear lisp is simple to parse, and has no annoying statements to hold > you back! > > Seriously though, there is an advantage to basing so much on > statements rather than expressions. We're specialized for one > statement per line, which is the most common case, and it allowed us > to have extraneous semicolons, braces, or whatever. Readability > benefits, style consistency benefits. > > Now there are some use cases that suffer here, including the one you > just gave: defining a dispatch dict with the functions inline. The > best I could do is define the dict first, then stick a decorator on > each function to register them. That's still ugly though. A creative > solution is needed, but none come to mind. class dispatcher(dispatch_dictionary): def x(value): ... def y(value): ... def int_23(value): ... With the proper dispatch_dictionary base class and it's metaclass, dispatcher would become a dictionary with functions that map from strings and integers to function handlers. Never underestimate the power of classes and metaclasses ;) . - Josiah From cliff at develix.com Thu Sep 11 03:14:25 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 10 Sep 2008 18:14:25 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> Message-ID: <1221095665.12242.183.camel@portable-evil> On Wed, 2008-09-10 at 17:16 -0600, Adam Olsen wrote: > On Wed, Sep 10, 2008 at 4:39 PM, Cliff Wells wrote: > > On Wed, 2008-09-10 at 15:57 -0600, Adam Olsen wrote: > >> On Wed, Sep 10, 2008 at 3:18 PM, Cliff Wells wrote: > >> > Further, I feel that this limitation forces programmers into using hacks > >> > and magic or overly spread-out code, which itself leads to readability > >> > concerns. Having used Python for around a decade, I'm quite aware of > >> > the fact that you can happily write tons and tons of nice code with > >> > Python in its current state. However, because of the direction of my > >> > work (developing a internal DSL in Python) I've suddenly become aware of > >> > this glass ceiling. I'd bumped into it before back when I was doing a > >> > lot of GUI development, but blamed it on lambda rather than realizing > >> > that it wasn't lambda so much as what I am bringing up now. > >> > >> Python is not intended for DSLs. Really, don't do it. Python is for > >> python code. > > > > The DSL I work on *is* Python code. And really, this is the first time > > I've heard anyone assert anything like this. Python is a > > general-purpose language. It's not VBA ;-) > > > > DSL's are an extremely useful concept. To summarily dispatch the whole > > of them with such an assertion is pretty much bolstering my argument: > > you've just asserted that Python is inherently limited in scope. > > I agree, DSL's are useful. They're just not something python > currently supports well. Well, this is where we agree. Where we seem to disagree is whether or not something needs to be done about it =) At some level, the whole concept of OO programming *is* DSL support. I consider "DSL" to mean "mapping of abstract (language) constructs to real-world (domain-specific) constructs". This type of DSL construction is quite-well supported by Python (and, in fact, is the way Breve is constructed under the hood). However, this support is hampered (to some degree) by having a somewhat inflexible syntax. > They're the use-case you need to justify > against the substantial changes you propose. I guess I don't see it as substantial to people who don't wish to use it (although it's quite substantial to people who do). Overall, I think this is why I feel the change doesn't require a huge amount of justification: you aren't *forced* to use it, but if you need it, it's huge. It doesn't impose any significant stylistic change on people who prefer the imperative style, but it opens vast doors for people wishing to approach problems from a functional path. > Maybe it's worth it, or > maybe it's better to add an indent-sensitive string literal that would > then allow *arbitrary* DSL syntax? Hm, I'd have to see an example (even contrived) of what you mean here. > > >> If you want another language, write your own parser. I > >> hear lisp is simple to parse, and has no annoying statements to hold > >> you back! > > > > Ah, except Python is the language I like in every way, *except* for this > > one particular wart. Really, had I not entered new programming domains > > and tried to take Python with me, I'd probably never have had a > > complaint. > > > > Also, external parsers defeat the entire reasoning behind internal DSL's > > (taking advantage of an established VM/compiler, requiring users to > > learn a new syntax in addition to their primary programming language). > > I appreciate what you're saying here, and feel much the same way about > my own pet-features, but this is a really poor argument. Everybody > has just one wafer-thin feature they'd like to add. Again, this is only wafer-thin if you don't want to use it (and that is part of its appeal - it's actually rather non-intrusive to classical statement-oriented programming). What I see happening in Python is exactly what you appear to be arguing against. Little specialized features are added one after the other to satisfy particular needs, when what is actually needed is one sweeping change that would make those features redundant. > >> Seriously though, there is an advantage to basing so much on > >> statements rather than expressions. We're specialized for one > >> statement per line, which is the most common case, > > > > Clearly it's the most common case in existing Python code since nothing > > else is allowed. But frankly, even Javascript doesn't follow this > > idiom anymore. Expression-oriented languages have seen a rebirth for > > good reasons (although I admit I'm none-to-fond of many of them, for > > various reasons). > > Again, "XYZ language has it" is an ineffective argument. Not really what I was trying to say. What I was saying is that expression-oriented languages are rising (practically from the grave C put them in) because they are inherently useful and typically more powerful than their imperative counterparts. Where they've tended to suffer is the place where Python shines: readability. What I'm claiming is that Python's readability is not due to its imperative nature, rather due to it's whitespace-oriented syntax and lack of line-noise, so Python could literally become the best of both worlds were it to shed its imperative roots. > >> and it allowed us > >> to have extraneous semicolons, braces, or whatever. > > > > Not following this. You mean to *not* have extraneous syntax? > > Consider a loop in C vs in python > > for (i = 0; i < 50; i++) { > a(); > b(); > } > > for i in range(50): > a() > b() > > We avoid the semicolons and braces because the newline and indentation > indicate the same thing. That's what I thought you meant. You dropped the "not" in your original statement. > That's what a statement it. Perhaps you can > retain this with your proposal, but only because a statement is still > the default, with expression as a rarely used option. Exactly. My point remains that the imperative *style* might still be preferred (and encouraged) because it *is* inherently simpler to understand, but that it shouldn't be enforced because it's also inherently limiting (not surprisingly). I think the community as a whole has been successful in forwarding "Pythonic" idioms even when the language allows "unPythonic" coding without resorting to b&d tactics within the language itself. I think the same external discipline can be applied to this concept without limiting Python's applicability in other domains. > >> Readability benefits, style consistency benefits. > > > > I strongly disagree. The artificial distinction between statements and > > expressions is the definition of *inconsistent*. Why do we have two > > versions of the "if" conditional? Why do we have "for" loops *and* list > > comprehensions? They express the same ideas, but the limitations in one > > required growing the language another direction. In short, we could > > have had a single, more powerful construct in place of two lesser > > constructs and simultaneously had less syntax to memorize and more > > consistency across the board. > > I was referring to consistency of the programs, not the language > itself. No silly arguments about brace position because they *are no > braces*. But you are assuming that eliminating statements somehow requires extra syntax. I think Logix, which was mentioned previously, adequately demonstrates that this is not the case. The only "extra" syntax that would need to be used would be parentheses for logical grouping of expressions. This syntax already exists in Python and is already used for the same. > There's actually a problem with trying to merge a for-loop and a list > comprehension. A generator expression is the canonical generic form, > but [genexp] would create a list containing a single genexp object. > > Likewise, a for-loop would become lazy, so without some extra syntax > to evaluate it (and trigger the side effects!), your programs would > cease to do anything. I don't think so. Think about how Python currently defines a generator: the presence of the "yield" keyword within a function. I think this same logic could be applied to a for loop (but I'm willing to be corrected if you see a problem). A for loop without "yield" is more or less what it is today (except it might evaluate to []). A for loop with "yield" is a generator (and by extension, useful as an expression). > So you see, despite significant and obvious similarity between the > features, there's some important differences as well. These are so > obvious from the contexts that you don't even think of them, so > clearly the mental load of having 3 (soon to be 5) different forms of > looping is not all that great. > > Mental load is what really counts, not some abstract concept of complexity. This isn't abstract. It's a matter of countable constructs, rules, and exceptions to rules. Consider: Rules in statement-oriented Python: 1) distinguishes between statements and expressions 2) expressions return a value, statements do not 3) expressions can be used inside other expressions and statements, statements cannot 4) there is an if statement 5) there is an if expression (ternary if operator) 6) there are for loop statements 7) there are list comprehensions 8) there are generators ("yield" defines when a function is a generator) Equivalent rules in expression-oriented Python: 1) if expression returns a value (that can be ignored or used in an expression) 2) for expression returns an empty list or a list if it's a generator (has "yield" keyword) that can be ignored or used in an expression. Which has more "mental load"? > >> Now there are some use cases that suffer here, including the one you > >> just gave: defining a dispatch dict with the functions inline. The > >> best I could do is define the dict first, then stick a decorator on > >> each function to register them. That's still ugly though. A creative > >> solution is needed, but none come to mind. > > > > That's because there is none. And this is my fundamental problem: it's > > not so much that it's hard to do in Python, it's that you *cannot* do it > > in Python. No amount of creativity, time, or experience will help, and > > this is disappointing. > > I didn't mean to do it in Python. I meant to modify the language. I don't consider modifying the language an acceptable solution for most programmers. It's a maintenance nightmare. > >> An example where this has happened before is the with-statement, which > >> is spectacularly successful IMO. Now, you may notice it could have > >> been done in a library rather than core syntax if generic anonymous > >> blocks were allowed ? so what? The library is still part of the > >> language! It's still something that has to be learned. And the > >> syntax would be substantially uglier using a generic mechanism, rather > >> than the specialized with-statement syntax. > > > > The "so what" is that it could *only* be implemented by the core devs. > > It was not possible for an average (or even above-average) Python > > programmer to write such a library, whereas it *could* have been had the > > language not prohibited it. > > As important as it is to extend the language via a library, somewhere > you need to draw the line and start modifying the syntax or other > fundamental builtins. Yes and no. I believe it should be possible to prototype almost any construct via a library. Whether the language should then embrace the concept embodied in that prototype to provide better integration, performance, or simply syntactic sugar, can then be argued much more fruitfully. If a language prevents you from creating such prototypes then I think > The is a universal tenant, applying even to > lisp (which has very little syntax, rather than extensible syntax). > > This is why we have a+b, rather than add(a, b). More syntax when it's worth it. But the language didn't prevent you from creating the hypothetical add() function. It merely provided syntactic sugar for making it more readable. I feel we're getting slightly side-tracked here =) > So it's not that we don't want to allow extensibility - quite the > opposite. It's that we want the common statement to be simpler, and > the extra syntax hasn't been justified for your use cases. Once again, I am forwarding *zero* extra syntax. In fact I am suggesting *less* syntax overall and an accompanying reduction in rules regarding the remaining syntax. I am suggesting removing (or deprecating) syntactic additions such as the ternary operator in favor of extending the power (or more to the point, removing arbitrary limitations) of the existing core language. Regards, Cliff From josiah.carlson at gmail.com Thu Sep 11 03:30:48 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Wed, 10 Sep 2008 18:30:48 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221095665.12242.183.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <1221095665.12242.183.camel@portable-evil> Message-ID: On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells wrote: > On Wed, 2008-09-10 at 17:16 -0600, Adam Olsen wrote: [snip] >> They're the use-case you need to justify >> against the substantial changes you propose. > > I guess I don't see it as substantial to people who don't wish to use it > (although it's quite substantial to people who do). Overall, I think > this is why I feel the change doesn't require a huge amount of > justification: you aren't *forced* to use it, but if you need it, it's > huge. It doesn't impose any significant stylistic change on people who > prefer the imperative style, but it opens vast doors for people wishing > to approach problems from a functional path. Once language syntax is added to, changed, etc., it's very difficult to remove those additions, changes, etc., even when the feature is rarely used, ugly, and generally a bad idea (see back-quotes `x` for repr(x) ). This may not seem like a big deal to you, because you want this feature, but for the rest of us who have little (arguably no) use for the feature, adding semantics to syntax, or adding syntax = additional mental overhead; never mind the additional sections in the tutorial where we have to explain why this *particular* special case of a DSL was special enough to break the rules of explicit is better than implicit (why should a multi-line if statement implicitly return something in one place but not in another?) I know, "it's just one little change". I've made the argument myself. But that doesn't mean that my idea was a good idea (it wasn't), nor does it mean that your current idea is (I think everyone in this thread except you would agree that it's a bad idea). Before continuing on in defending your proposal here, I suggest you try comp.lang.python . If you can't get a dozen people to agree that it's a good idea, or if (like here) the only replies are negative (or even if the majority of them are), then I would suggest you drop it. - Josiah From rhamph at gmail.com Thu Sep 11 04:24:07 2008 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 10 Sep 2008 20:24:07 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221095665.12242.183.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <1221095665.12242.183.camel@portable-evil> Message-ID: On Wed, Sep 10, 2008 at 7:14 PM, Cliff Wells wrote: > On Wed, 2008-09-10 at 17:16 -0600, Adam Olsen wrote: >> On Wed, Sep 10, 2008 at 4:39 PM, Cliff Wells wrote: >> > On Wed, 2008-09-10 at 15:57 -0600, Adam Olsen wrote: >> >> On Wed, Sep 10, 2008 at 3:18 PM, Cliff Wells wrote: >> >> > Further, I feel that this limitation forces programmers into using hacks >> >> > and magic or overly spread-out code, which itself leads to readability >> >> > concerns. Having used Python for around a decade, I'm quite aware of >> >> > the fact that you can happily write tons and tons of nice code with >> >> > Python in its current state. However, because of the direction of my >> >> > work (developing a internal DSL in Python) I've suddenly become aware of >> >> > this glass ceiling. I'd bumped into it before back when I was doing a >> >> > lot of GUI development, but blamed it on lambda rather than realizing >> >> > that it wasn't lambda so much as what I am bringing up now. >> >> >> >> Python is not intended for DSLs. Really, don't do it. Python is for >> >> python code. >> > >> > The DSL I work on *is* Python code. And really, this is the first time >> > I've heard anyone assert anything like this. Python is a >> > general-purpose language. It's not VBA ;-) >> > >> > DSL's are an extremely useful concept. To summarily dispatch the whole >> > of them with such an assertion is pretty much bolstering my argument: >> > you've just asserted that Python is inherently limited in scope. >> >> I agree, DSL's are useful. They're just not something python >> currently supports well. > > Well, this is where we agree. Where we seem to disagree is whether or > not something needs to be done about it =) > > At some level, the whole concept of OO programming *is* DSL support. I > consider "DSL" to mean "mapping of abstract (language) constructs to > real-world (domain-specific) constructs". This type of DSL construction > is quite-well supported by Python (and, in fact, is the way Breve is > constructed under the hood). However, this support is hampered (to > some degree) by having a somewhat inflexible syntax. Aye, but.. it's hard to explain. There's different levels upon which you define a language. At the lowest extreme a crude language with no functions will show patterns in its structure, due to how you use it. At the highest extreme you change how the underlying parser works, or maybe even use a graphical language instead. In between is a balance of how predictable the language is (defining new functions rather than a new way to *call* functions), with how powerful it is. You can't simply have both. If the syntax is too open you may as well stick a new parser in every file, and hope they learn to read it. >> They're the use-case you need to justify >> against the substantial changes you propose. > > I guess I don't see it as substantial to people who don't wish to use it > (although it's quite substantial to people who do). Overall, I think > this is why I feel the change doesn't require a huge amount of > justification: you aren't *forced* to use it, but if you need it, it's > huge. It doesn't impose any significant stylistic change on people who > prefer the imperative style, but it opens vast doors for people wishing > to approach problems from a functional path. > >> Maybe it's worth it, or >> maybe it's better to add an indent-sensitive string literal that would >> then allow *arbitrary* DSL syntax? > > Hm, I'd have to see an example (even contrived) of what you mean here. Today, you'd have to use something like this: foo(""" bar """) It's ugly. It uses the equivalent of braces to tell the parser something that's obvious to us from the indentation. We want it to look more like a normal block: foo: bar However, this is no longer a function call. You can't put a value inside parenthesis without moving the parenthesis to the last line, which is what we don't want. Adding syntax to the language is the normal solution, but we're looking for something open-ended (that embeds a string, not python code). We have to cheat somehow. One way is borrow from decorators: @foo blob: bar Another is to use $blob$ as a placeholder (only legal when the line is just an expression, not a statement): foo($blob$): bar >> >> If you want another language, write your own parser. I >> >> hear lisp is simple to parse, and has no annoying statements to hold >> >> you back! >> > >> > Ah, except Python is the language I like in every way, *except* for this >> > one particular wart. Really, had I not entered new programming domains >> > and tried to take Python with me, I'd probably never have had a >> > complaint. >> > >> > Also, external parsers defeat the entire reasoning behind internal DSL's >> > (taking advantage of an established VM/compiler, requiring users to >> > learn a new syntax in addition to their primary programming language). >> >> I appreciate what you're saying here, and feel much the same way about >> my own pet-features, but this is a really poor argument. Everybody >> has just one wafer-thin feature they'd like to add. > > Again, this is only wafer-thin if you don't want to use it (and that is > part of its appeal - it's actually rather non-intrusive to classical > statement-oriented programming). > > What I see happening in Python is exactly what you appear to be arguing > against. Little specialized features are added one after the other to > satisfy particular needs, when what is actually needed is one sweeping > change that would make those features redundant. Such a sweeping change would only move them into the library, not remove them from the language. Indeed, because they're forced into an awkward over-generalized syntax, they become significantly harder to use. >> >> Seriously though, there is an advantage to basing so much on >> >> statements rather than expressions. We're specialized for one >> >> statement per line, which is the most common case, >> > >> > Clearly it's the most common case in existing Python code since nothing >> > else is allowed. But frankly, even Javascript doesn't follow this >> > idiom anymore. Expression-oriented languages have seen a rebirth for >> > good reasons (although I admit I'm none-to-fond of many of them, for >> > various reasons). >> >> Again, "XYZ language has it" is an ineffective argument. > > Not really what I was trying to say. What I was saying is that > expression-oriented languages are rising (practically from the grave C > put them in) because they are inherently useful and typically more > powerful than their imperative counterparts. Where they've tended to > suffer is the place where Python shines: readability. What I'm > claiming is that Python's readability is not due to its imperative > nature, rather due to it's whitespace-oriented syntax and lack of > line-noise, so Python could literally become the best of both worlds > were it to shed its imperative roots. Most programmers don't get it. They'd think reducing the axioms in math from 10 to 5 would make it simpler to learn algebra. Not only would this have no effect on how they use algebra, they probably can't even list the axioms anyway! >> There's actually a problem with trying to merge a for-loop and a list >> comprehension. A generator expression is the canonical generic form, >> but [genexp] would create a list containing a single genexp object. >> >> Likewise, a for-loop would become lazy, so without some extra syntax >> to evaluate it (and trigger the side effects!), your programs would >> cease to do anything. > > I don't think so. Think about how Python currently defines a generator: > the presence of the "yield" keyword within a function. I think this > same logic could be applied to a for loop (but I'm willing to be > corrected if you see a problem). A for loop without "yield" is more or > less what it is today (except it might evaluate to []). A for loop with > "yield" is a generator (and by extension, useful as an expression). You're thinking about a generator function, not a generator expression. A generator expression is the same syntax as a list comprehension, but it uses () rather rather than []. Unless you meant to change how generator expressions work.. but that obviously wouldn't be compatible (nor would any change to a for-statement). >> So you see, despite significant and obvious similarity between the >> features, there's some important differences as well. These are so >> obvious from the contexts that you don't even think of them, so >> clearly the mental load of having 3 (soon to be 5) different forms of >> looping is not all that great. >> >> Mental load is what really counts, not some abstract concept of complexity. > > This isn't abstract. It's a matter of countable constructs, rules, and > exceptions to rules. Consider: > > Rules in statement-oriented Python: > > 1) distinguishes between statements and expressions > 2) expressions return a value, statements do not > 3) expressions can be used inside other expressions and statements, > statements cannot > 4) there is an if statement > 5) there is an if expression (ternary if operator) > 6) there are for loop statements > 7) there are list comprehensions > 8) there are generators ("yield" defines when a function is a generator) > > Equivalent rules in expression-oriented Python: > > 1) if expression returns a value (that can be ignored or used in an > expression) > 2) for expression returns an empty list or a list if it's a generator > (has "yield" keyword) that can be ignored or used in an expression. > > Which has more "mental load"? You're assuming statements and expressions won't continue to be done in different styles, which just isn't true. Even though a statement *could* return a value, most of the time it would be used as if it doesn't, so it remains as a special case to be remembered. There's also at least two different modes for a for-loop (lazy vs eager), assuming you drop the list-comp (and 3.0's set/dict comprehensions). That's what I meant by "abstract concept of complexity", although maybe I need a better label for it. You've recategorized thinks to be the same, yet we're expected to use the same way we always did (other than occasionally using new functionality). The categories don't determine complexity, how we use them does! Your changes actually add a substantial amount of complexity, as well as being unimplementable (ambiguous or incompatible behaviour). They never had a chance of being accepted, but I'm trying to explain why they don't work, so you and other budding language developers might give better suggestions in the future. >> >> Now there are some use cases that suffer here, including the one you >> >> just gave: defining a dispatch dict with the functions inline. The >> >> best I could do is define the dict first, then stick a decorator on >> >> each function to register them. That's still ugly though. A creative >> >> solution is needed, but none come to mind. >> > >> > That's because there is none. And this is my fundamental problem: it's >> > not so much that it's hard to do in Python, it's that you *cannot* do it >> > in Python. No amount of creativity, time, or experience will help, and >> > this is disappointing. >> >> I didn't mean to do it in Python. I meant to modify the language. > > I don't consider modifying the language an acceptable solution for most > programmers. It's a maintenance nightmare. If you're not willing to solve it properly then don't come whining to us. Some problems *need* drastic solutions. >> >> An example where this has happened before is the with-statement, which >> >> is spectacularly successful IMO. Now, you may notice it could have >> >> been done in a library rather than core syntax if generic anonymous >> >> blocks were allowed ? so what? The library is still part of the >> >> language! It's still something that has to be learned. And the >> >> syntax would be substantially uglier using a generic mechanism, rather >> >> than the specialized with-statement syntax. >> > >> > The "so what" is that it could *only* be implemented by the core devs. >> > It was not possible for an average (or even above-average) Python >> > programmer to write such a library, whereas it *could* have been had the >> > language not prohibited it. >> >> As important as it is to extend the language via a library, somewhere >> you need to draw the line and start modifying the syntax or other >> fundamental builtins. > > Yes and no. I believe it should be possible to prototype almost any > construct via a library. Whether the language should then embrace the > concept embodied in that prototype to provide better integration, > performance, or simply syntactic sugar, can then be argued much more > fruitfully. If a language prevents you from creating such prototypes > then I think Depends how ugly you're willing to let it get. There's many ways to do a dispatcher dict, a couple of which have been mentioned already. -- Adam Olsen, aka Rhamphoryncus From cliff at develix.com Thu Sep 11 06:34:51 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 10 Sep 2008 21:34:51 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <1221095665.12242.183.camel@portable-evil> Message-ID: <1221107691.12242.289.camel@portable-evil> On Wed, 2008-09-10 at 18:30 -0700, Josiah Carlson wrote: > On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells wrote: > > On Wed, 2008-09-10 at 17:16 -0600, Adam Olsen wrote: > [snip] > >> They're the use-case you need to justify > >> against the substantial changes you propose. > > > > I guess I don't see it as substantial to people who don't wish to use it > > (although it's quite substantial to people who do). Overall, I think > > this is why I feel the change doesn't require a huge amount of > > justification: you aren't *forced* to use it, but if you need it, it's > > huge. It doesn't impose any significant stylistic change on people who > > prefer the imperative style, but it opens vast doors for people wishing > > to approach problems from a functional path. > > Once language syntax is added to, changed, etc., it's very difficult > to remove those additions, changes, etc., even when the feature is > rarely used, ugly, and generally a bad idea (see back-quotes `x` for > repr(x) ). This may not seem like a big deal to you, because you want > this feature, but for the rest of us who have little (arguably no) use > for the feature, adding semantics to syntax, or adding syntax = > additional mental overhead; Again I assert the opposite: that Python is currently forced to explain the arbitrary distinction that currently exists and that a significant amount of extra syntax and language features exist to work around that distinction. People may not clamor for the change but there's quite a few newcomers to Python who must have the distinction explained to them. > never mind the additional sections in the > tutorial where we have to explain why this *particular* special case > of a DSL was special enough to break the rules of explicit is better > than implicit (why should a multi-line if statement implicitly return > something in one place but not in another?) I respect your work and knowledge of Python, but that's insufficient to get me to respect a straw-man argument. I've not suggested that Python add anything for any particular situation. I'm suggesting a change that would make it more logically consistent, expressive, and help reduce the language and syntax clutter it's been accruing at a steady pace. The fact that it also helps in a rather broad range of problems is ancillary and my particular use case was presented as an example, not as *the* problem to be solved. Also, I don't know where you got the idea that I suggest any expression should return a value sometimes but not others. I'm suggesting they *always* return values, but that you don't always need to worry about them unless you plan to use the value in another expression. Kind of how functions already work in Python. > I know, "it's just one little change". I've made the argument myself. > But that doesn't mean that my idea was a good idea (it wasn't), nor > does it mean that your current idea is (I think everyone in this > thread except you would agree that it's a bad idea). Sure. Because everyone who might have agreed with me has already migrated to Ruby or started a new language (Logix, Boo, et al). I just happen to be lazy enough to entertain the thin hope that Python could be fixed so I don't have to learn a new language or programming environment ;-) > Before continuing on in defending your proposal here, I suggest you > try comp.lang.python . If you can't get a dozen people to agree that > it's a good idea, or if (like here) the only replies are negative (or > even if the majority of them are), then I would suggest you drop it. Actually I avoided c.l.py because I saw this list suggested as the appropriate place for new or controversial ideas (and the list name seemed to suggest the same). The fact that lambda was so bitterly defended against GvR's strong desire to remove it should be hint enough that a significant number of Pythonistas are interested in functional and expression-oriented programming. In any case, I actually got the response I expected and ultimately I expect the discussion here was probably far more enlightened than I would expect on c.l.py. It seems most Pythoneers work in particular domains where this isn't an issue, are satisfied with the workarounds, or simply are unable to see there's a vast world of elegant solutions beyond what imperative languages can describe. Unfortunately this only confirms my (rather reluctant) expectation that I'll be forced to move to a more expressive language to satisfy my programming itches. In any case, thanks for the feedback. Regards, Cliff From george.sakkis at gmail.com Thu Sep 11 07:39:55 2008 From: george.sakkis at gmail.com (George Sakkis) Date: Thu, 11 Sep 2008 01:39:55 -0400 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221107691.12242.289.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <1221095665.12242.183.camel@portable-evil> <1221107691.12242.289.camel@portable-evil> Message-ID: <91ad5bf80809102239q3edc4dcav8fa216f48467c5fe@mail.gmail.com> On Thu, Sep 11, 2008 at 12:34 AM, Cliff Wells wrote: On Wed, 2008-09-10 at 18:30 -0700, Josiah Carlson wrote: > > On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells wrote: > > > On Wed, 2008-09-10 at 17:16 -0600, Adam Olsen wrote: > > [snip] > > >> They're the use-case you need to justify > > >> against the substantial changes you propose. > > > > > > I guess I don't see it as substantial to people who don't wish to use > it > > > (although it's quite substantial to people who do). Overall, I think > > > this is why I feel the change doesn't require a huge amount of > > > justification: you aren't *forced* to use it, but if you need it, it's > > > huge. It doesn't impose any significant stylistic change on people who > > > prefer the imperative style, but it opens vast doors for people wishing > > > to approach problems from a functional path. > > > > Once language syntax is added to, changed, etc., it's very difficult > > to remove those additions, changes, etc., even when the feature is > > rarely used, ugly, and generally a bad idea (see back-quotes `x` for > > repr(x) ). This may not seem like a big deal to you, because you want > > this feature, but for the rest of us who have little (arguably no) use > > for the feature, adding semantics to syntax, or adding syntax = > > additional mental overhead; > > Again I assert the opposite: that Python is currently forced to explain > the arbitrary distinction that currently exists and that a significant > amount of extra syntax and language features exist to work around that > distinction. People may not clamor for the change but there's quite a > few newcomers to Python who must have the distinction explained to them. For better or for worse, most people come to Python with imperative rather than functional language background, so the distinction is either not even realized or it usually seems "natural" when realized. Personally, I find some features to be more natural as statements (e.g. it is typically obvious whether to use a "for" loop or a list/gen. comprehension) but others unnecessarily limiting (e.g. having to write "lambda x: x.__setitem__(0,1)" instead of "lambda x: x[0]=1"). George -------------- next part -------------- An HTML attachment was scrubbed... URL: From josiah.carlson at gmail.com Thu Sep 11 07:47:17 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Wed, 10 Sep 2008 22:47:17 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221107691.12242.289.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <1221095665.12242.183.camel@portable-evil> <1221107691.12242.289.camel@portable-evil> Message-ID: On Wed, Sep 10, 2008 at 9:34 PM, Cliff Wells wrote: > On Wed, 2008-09-10 at 18:30 -0700, Josiah Carlson wrote: >> On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells wrote: >> > On Wed, 2008-09-10 at 17:16 -0600, Adam Olsen wrote: >> [snip] >> >> They're the use-case you need to justify >> >> against the substantial changes you propose. >> > >> > I guess I don't see it as substantial to people who don't wish to use it >> > (although it's quite substantial to people who do). Overall, I think >> > this is why I feel the change doesn't require a huge amount of >> > justification: you aren't *forced* to use it, but if you need it, it's >> > huge. It doesn't impose any significant stylistic change on people who >> > prefer the imperative style, but it opens vast doors for people wishing >> > to approach problems from a functional path. >> >> Once language syntax is added to, changed, etc., it's very difficult >> to remove those additions, changes, etc., even when the feature is >> rarely used, ugly, and generally a bad idea (see back-quotes `x` for >> repr(x) ). This may not seem like a big deal to you, because you want >> this feature, but for the rest of us who have little (arguably no) use >> for the feature, adding semantics to syntax, or adding syntax = >> additional mental overhead; > > Again I assert the opposite: that Python is currently forced to explain > the arbitrary distinction that currently exists and that a significant > amount of extra syntax and language features exist to work around that > distinction. People may not clamor for the change but there's quite a > few newcomers to Python who must have the distinction explained to them. For people who have gone through the Python tutorial or any one of the many good 'learning Python' texts (online, paper, free, and paid-for) the distinction has already been explained. For those who refuse (for some reason) to go through one of those resources, that's not an issue with Python, that's an issue with the user. As a data point; everyone that I've taught Python over the years (undergrads and interns who only ever used Java before to 30-year programming veterans of apl/cobol/fortran), not a single one of them has ever had a problem with the difference between statements and expressions. It is possible that I've only ever worked with gifted students and coworkers, though this is unlikely. >> never mind the additional sections in the >> tutorial where we have to explain why this *particular* special case >> of a DSL was special enough to break the rules of explicit is better >> than implicit (why should a multi-line if statement implicitly return >> something in one place but not in another?) > > I respect your work and knowledge of Python, but that's insufficient to > get me to respect a straw-man argument. I've not suggested that Python > add anything for any particular situation. I'm suggesting a change that > would make it more logically consistent, expressive, and help reduce the > language and syntax clutter it's been accruing at a steady pace. The > fact that it also helps in a rather broad range of problems is ancillary > and my particular use case was presented as an example, not as *the* > problem to be solved. What I've found (historically) is that any time I find myself trying to do something that is not available within the syntax of Python, it's usually because it's a bad idea. In your case, you want (for example) the following to be valid Python... lfcn = lambda x: ( if x: True else: False) Similarly... lfcn = lambda x: ( for i in xrange(x): yield fcn(i, x, ofcn(i)) ) Why? Initially it is for a templating language that you have designed that makes these kinds of things difficult to do during an automatic translation from working Python source code into compiled source code. Presumably something of the form... html [ body [ a (href='http://python.org') ["Python!"] ] ] To be converted into something like... def generate_html(out): out.write(''' Python! ''') Trust me, I understand your desire. I have used Cheetah and Spitfire (http://code.google.com/p/spitfire/), and the speedups gained by translating templates into Python source are not to be laughed at. And before I used those, I saw nevow.stan (from which your syntax is effectively identical to), and wrote my own variant (http://code.activestate.com/recipes/440563/ - I use multi-calling semantics rather than __getitem__ stuff). The only reason I didn't write a compiler (and run into the same issues you have) was because I didn't care about dynamic content generation, I was using it for pre-generation of static web pages. But I digress. At some point, you acknowledge that your templating language is constrained by your programming language of choice (as is the case for systems that use the base programming language for syntax, because of the convenience of parsing), or really, your templating language is constrained because you refuse to write your own parser. Don't get me wrong, the convenience of writing a templating language in Python syntax is amazing...but you've noticed that your compilation steps are hampered by Python's immediate execution of code. In fact, your own renderers are a work-around for something that is /relatively/ natural in Cheetah/Spitfire: $for $link in $links: $link.label Which is generally compiled into the following (with a bit of extra garbage, but you get the idea)... for link in links: out.write('''%s'''%(link.url, link.label)) Now, I loathe writing Cheetah/Spitfire because I loathe writing html, which is why I wrote my own templating system, but there are work-arounds. For example, if you want to stick with Python syntax... For("$link", "$links", "$link.label Also, I don't know where you got the idea that I suggest any expression > should return a value sometimes but not others. I'm suggesting they > *always* return values, but that you don't always need to worry about > them unless you plan to use the value in another expression. Kind of > how functions already work in Python. You are right, I misread an earlier email in this thread. >> I know, "it's just one little change". I've made the argument myself. >> But that doesn't mean that my idea was a good idea (it wasn't), nor >> does it mean that your current idea is (I think everyone in this >> thread except you would agree that it's a bad idea). > > Sure. Because everyone who might have agreed with me has already > migrated to Ruby or started a new language (Logix, Boo, et al). I just > happen to be lazy enough to entertain the thin hope that Python could be > fixed so I don't have to learn a new language or programming > environment ;-) That's not necessary. You've already solved your problem with renderers, and I've pointed out another method for getting some behavior in-line (If you are careful, you could probably build all of Python out of constructs similar to the For above). Of course there's always the option of just using some version of the output of the compiler/ast (I have done as much to pull out class definitions, function definitions, etc., in the editor that I have written). You could even write a tool for doing the conversion ;) . >> Before continuing on in defending your proposal here, I suggest you >> try comp.lang.python . If you can't get a dozen people to agree that >> it's a good idea, or if (like here) the only replies are negative (or >> even if the majority of them are), then I would suggest you drop it. > > Actually I avoided c.l.py because I saw this list suggested as the > appropriate place for new or controversial ideas (and the list name > seemed to suggest the same). > > The fact that lambda was so bitterly defended against GvR's strong > desire to remove it should be hint enough that a significant number of > Pythonistas are interested in functional and expression-oriented > programming. Indeed! I have a penchant for the use of list comprehensions, generator expressions, map, etc. - when they are reasonably applicable. The question is always: what is reasonably applicable. > In any case, I actually got the response I expected and ultimately I > expect the discussion here was probably far more enlightened than I > would expect on c.l.py. > > It seems most Pythoneers work in particular domains where this isn't an > issue, are satisfied with the workarounds, or simply are unable to see > there's a vast world of elegant solutions beyond what imperative > languages can describe. Unfortunately this only confirms my (rather > reluctant) expectation that I'll be forced to move to a more expressive > language to satisfy my programming itches. Pythoneers do see that there are /alternate/ solutions to what is currently available within Python, hence why people tend to like to write parsers in Python ;) . One thing you may want to look into is that you aren't using the full expressive power of Python in your templating language, and because of this, it is actually quite easy to write a parser for your subset, adding all of the necessary behavior and translation. I used DParser in Python for a few years and loved it, though it seems like the home page for it is no longer available. On the one hand, yes, you are currently constrained by the limits of Python's syntax. And yes, you could move to Ruby, Boo, etc. Or you could try alternate constructs (For, If, Def, etc.), write your own parser (which isn't hard), or shrug and admit to yourself that the limitations really aren't all that bad. Inconvenient, sure. The end of breve in Python? Not necessarily. - Josiah From szport at gmail.com Thu Sep 11 09:53:55 2008 From: szport at gmail.com (Zaur Shibzoukhov) Date: Thu, 11 Sep 2008 11:53:55 +0400 Subject: [Python-ideas] Fwd: About calling syntax In-Reply-To: <48C8252D.8070004@latte.ca> References: <48C8252D.8070004@latte.ca> Message-ID: Returning to initial topic of my posts I said about calling function in the case: class Element(object): def __init__(self, name, *args, **kw): self.__dict__.update(kw) self.chidren.extend(args) I tried to find more natural form of initializing such tree-like structures in python. In current syntax the only available form (which is natural to me) is: Element('root', x='1', y='2', *[ Element('child1', x='1', y='2', *[ Element(''grandchild1', x='4', y='6'), Element(''grandchild2', x='7', y='8') ]), Element(''grandchild1', x='4', y='6') ]) I hoped that python calling syntax could be extended in order to allow writing such initializaing code without *[ ... ] Best regards. Zaur 2008/9/10 Blake Winton : > Zaur Shibzoukhov wrote: >> >> 2008/9/10 Bruce Leban > >> >> I'm sure there are scenarios where this is useful but I think in >> general it would be less readable. As it is, I can't miss a >> positional argument in the middle of a long list of keyword arguments: >> >> foo(a=b,c=d,e=f,g=h,i=j,k=l,m) >> >> >> But programmer could select most readable form of function call (as the >> case may be). > > But some of them wouldn't, and I would have to learn all the forms of > function call to read other people's code. Additional forms are easy for > the writer, but hard for the reader, and I read far more code than I write. > > To say it a different way, programmers _could_ write easy-to-understand > Perl, but they can also write hard-to-understand Perl, and in my experience > when presented with that choice, most of them choose the hard-to-understand > route. ;) > > Later, > Blake. > From g.brandl at gmx.net Thu Sep 11 11:00:03 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 11 Sep 2008 11:00:03 +0200 Subject: [Python-ideas] Fwd: About calling syntax In-Reply-To: References: <48C8252D.8070004@latte.ca> Message-ID: Zaur Shibzoukhov schrieb: > Returning to initial topic of my posts I said about calling function > in the case: > > class Element(object): > def __init__(self, name, *args, **kw): > self.__dict__.update(kw) > self.chidren.extend(args) > > I tried to find more natural form of initializing such tree-like > structures in python. > In current syntax the only available form (which is natural to me) is: > > Element('root', > x='1', y='2', *[ > Element('child1', > x='1', y='2', *[ > Element(''grandchild1', x='4', y='6'), > Element(''grandchild2', x='7', y='8') > ]), > Element(''grandchild1', x='4', y='6') > ]) > > I hoped that python calling syntax could be extended in order to allow > writing such initializaing code without *[ ... ] Element('root', x='1', y='2', c=[ Element('child1', x='1', y='2', c=[ Element(''grandchild1', x='4', y='6'), Element(''grandchild2', x='7', y='8') ]), Element(''grandchild1', x='4', y='6') ]) Voila! ;) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From scott+python-ideas at scottdial.com Thu Sep 11 11:02:05 2008 From: scott+python-ideas at scottdial.com (Scott Dial) Date: Thu, 11 Sep 2008 05:02:05 -0400 Subject: [Python-ideas] Fwd: About calling syntax In-Reply-To: References: <48C8252D.8070004@latte.ca> Message-ID: <48C8DE8D.9060004@scottdial.com> Zaur Shibzoukhov wrote: > In current syntax the only available form (which is natural to me) is: > > Element('root', > x='1', y='2', *[ > Element('child1', > x='1', y='2', *[ > Element(''grandchild1', x='4', y='6'), > Element(''grandchild2', x='7', y='8') > ]), > Element(''grandchild1', x='4', y='6') > ]) That doesn't look so bad. I wonder what's so bad with give "Element" an "add_child" method.. class Element(object): def __init__(self, name, **attrs): self.name = name self.attrs = attrs self.children = [] def add_child(*children): self.children.extend(children) return self Element('root', x='1', y='2').add_child( Element('child1', x='1', y='2').add_child( Element('grandchild1', x='4', y='6'), Element('grandchild2', x='7', y='8'), ), Element('child2', x='1', y='2'), ) Presumably you would want to have an add_child() already for other purposes? Unless there is a compelling reason to not make "add_child()" a composable function.. So, I don't see the need to start inventing syntax for such a simple pattern. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From cliff at develix.com Thu Sep 11 11:05:25 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 02:05:25 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <1221095665.12242.183.camel@portable-evil> <1221107691.12242.289.camel@portable-evil> Message-ID: <1221123925.12242.406.camel@portable-evil> On Wed, 2008-09-10 at 22:47 -0700, Josiah Carlson wrote: > On Wed, Sep 10, 2008 at 9:34 PM, Cliff Wells wrote: > > On Wed, 2008-09-10 at 18:30 -0700, Josiah Carlson wrote: > >> On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells wrote: > > Again I assert the opposite: that Python is currently forced to explain > > the arbitrary distinction that currently exists and that a significant > > amount of extra syntax and language features exist to work around that > > distinction. People may not clamor for the change but there's quite a > > few newcomers to Python who must have the distinction explained to them. > > For people who have gone through the Python tutorial or any one of the > many good 'learning Python' texts (online, paper, free, and paid-for) > the distinction has already been explained. For those who refuse (for > some reason) to go through one of those resources, that's not an issue > with Python, that's an issue with the user. You won't catch me disagreeing with this, but I was trying (clumsily) to point out that the distinction violates (albeit subtly) the principal of least surprise and so requires explanation (even if the explanation is very simple). > As a data point; everyone that I've taught Python over the years > (undergrads and interns who only ever used Java before to 30-year > programming veterans of apl/cobol/fortran), not a single one of them > has ever had a problem with the difference between statements and > expressions. It is possible that I've only ever worked with gifted > students and coworkers, though this is unlikely. Without any supporting data, my hunch is that most people who migrate to Python come from other imperative languages (myself included) and so don't find the distinction foreign. > What I've found (historically) is that any time I find myself trying > to do something that is not available within the syntax of Python, > it's usually because it's a bad idea. In your case, you want (for > example) the following to be valid Python... > > lfcn = lambda x: ( > if x: > True > else: > False) > > Similarly... > > lfcn = lambda x: ( > for i in xrange(x): > yield fcn(i, x, ofcn(i)) > ) > > Why? Initially it is for a templating language that you have designed > that makes these kinds of things difficult to do during an automatic > translation from working Python source code into compiled source code. > Presumably something of the form... > > html [ body [ a (href='http://python.org') ["Python!"] ] ] > > To be converted into something like... > > def generate_html(out): > out.write(''' > > Python! > > ''') Actually, it's much simpler than that. The HTML tags are objects which simply get serialized into their text representation. I specifically do not generate Python source. In fact, you can simply do something like this in the interactive interpreter: >>> print html [ body [ a (href='http://python.org') ["Python!"] ] ] >>> Python! This is more or less how Nevow Stan does it as well. > But I digress. At some point, you acknowledge that your templating > language is constrained by your programming language of choice (as is > the case for systems that use the base programming language for > syntax, because of the convenience of parsing), or really, your > templating language is constrained because you refuse to write your > own parser. Don't get me wrong, the convenience of writing a > templating language in Python syntax is amazing...but you've noticed > that your compilation steps are hampered by Python's immediate > execution of code. In fact, your own renderers are a work-around for > something that is /relatively/ natural in Cheetah/Spitfire: > > $for $link in $links: > $link.label > > Which is generally compiled into the following (with a bit of extra > garbage, but you get the idea)... > > for link in links: > out.write('''%s'''%(link.url, link.label)) > > Now, I loathe writing Cheetah/Spitfire because I loathe writing html, > which is why I wrote my own templating system, but there are > work-arounds. For example, if you want to stick with Python syntax... > > For("$link", "$links", "$link.label > Now you just need to write your For factory and flattener, which > produces the right code (easy), and with a little work, you can even > make for loops composable, add ifs, etc. I know, it's not as elegant > (for you) in this situation as statements becoming expressions, but it > will work today. (note that pie-in-the-sky best-case scenario is 18+ > months for Python 3.1, but more likely 15 years for Python 4 ;) ) I've actually employed several methods for working around this issue in Breve: For looping: - list comprehensions in place of for loops (arguably the natural choice in Python, but list comps tend toward unreadability quickly). - tag multiplication e.g. li(class_='$class')['$value'] * [{'class': 'even', 'value': 1 }, {'class': 'odd', 'value': 2}] results in
  • 1
  • 2
    • For conditionals: - ternary if-operator for Python >=2.5 - test(val) and [ block ] in place of if-statement (for <2.5 support) e.g. test (session.logged_in) and div['hello', session.user] For both: - renderers - push the logic back into Python proper (as you suggested) At one point, I even wrote a set of custom tag classes that allowed things like this: switch ( x ) [ case ( 1 ) [ 'x is 1' ], case ( 2 ) [ 'x is 2' ], case ( 3 ) [ 'x is 3' ], default [ 'x is not in list' ] ] While these worked superficially, they were ill-fated due to lack of short-circuiting/lazy evaluation (which rendered them both inefficient and incorrect). At the end of the day, while these sorts of methods work, they feel like what they are: workarounds (although 2.5 helps a bit with the ternary if-operator). This might be nit-picking on my part, but I don't think I'm especially unusual in my desire to do the "Right Thing". Ultimately though, whether/how I solved these issues can be considered two ways: from a practical point-of-view (what you are espousing) or from a purist's point-of-view (which I know is not a popular view amongst Pythonistas). I mentally weighed the pros and cons of having statements and ended up finding the pros lacking (ultimately coming down to enforcing an imperative style). > That's not necessary. You've already solved your problem with > renderers, and I've pointed out another method for getting some > behavior in-line (If you are careful, you could probably build all of > Python out of constructs similar to the For above). Of course there's > always the option of just using some version of the output of the > compiler/ast (I have done as much to pull out class definitions, > function definitions, etc., in the editor that I have written). You > could even write a tool for doing the conversion ;) . I've considered this, but frankly find AST manipulation a bit daunting (and even more of a workaround than the above pure-Python solutions). I've also considered trying out EasyExtend to accomplish my goals but it still doesn't feel quite right. > > The fact that lambda was so bitterly defended against GvR's strong > > desire to remove it should be hint enough that a significant number of > > Pythonistas are interested in functional and expression-oriented > > programming. > > Indeed! I have a penchant for the use of list comprehensions, > generator expressions, map, etc. - when they are reasonably > applicable. The question is always: what is reasonably applicable. As you might expect, I subscribe to the "consenting adult" perspective. I think "reasonably applicable" is a determination best left to the programmer managing a particular piece of code rather than the programming language. Readability matters most to those who have to read it (which is, more often than not, the person who wrote it). What is readable to one isn't always equally readable to someone else. For example, list comprehensions used to appear completely inside-out to me whereas map/filter/reduce was as simple as could be, and yet I've hear GvR claim on several occasions that the exact opposite is true for him. Most likely we'd differ just as much on any sufficiently non-trivial bit of code. But at the end of the day, GvR probably won't need to worry about how readable any bit of my code is, it only matters to me. And even if someone else were to need to read my code, it's a toss-up which way he'd find more readable. Ultimately I don't think there's a clear win here either way, but I feel that Python needlessly enforces a single vision to the exclusion of other equally valid visions. > > In any case, I actually got the response I expected and ultimately I > > expect the discussion here was probably far more enlightened than I > > would expect on c.l.py. > > > > It seems most Pythoneers work in particular domains where this isn't an > > issue, are satisfied with the workarounds, or simply are unable to see > > there's a vast world of elegant solutions beyond what imperative > > languages can describe. Unfortunately this only confirms my (rather > > reluctant) expectation that I'll be forced to move to a more expressive > > language to satisfy my programming itches. > > Pythoneers do see that there are /alternate/ solutions to what is > currently available within Python, hence why people tend to like to > write parsers in Python ;) . One thing you may want to look into is > that you aren't using the full expressive power of Python in your > templating language, and because of this, it is actually quite easy to > write a parser for your subset, adding all of the necessary behavior > and translation. I used DParser in Python for a few years and loved > it, though it seems like the home page for it is no longer available. Well, as far as Breve goes, I think there are adequate workarounds. What has disturbed me was finding I that I actually needed workarounds (something I haven't needed a lot of in my years of Python programming). This got me to thinking about the dichotomy of statements and expressions in Python and I was irrevocably drawn to the conclusion that this distinction is not as inconsequential as I had once believed. > On the one hand, yes, you are currently constrained by the limits of > Python's syntax. And yes, you could move to Ruby, Boo, etc. Or you > could try alternate constructs (For, If, Def, etc.), write your own > parser (which isn't hard), or shrug and admit to yourself that the > limitations really aren't all that bad. Inconvenient, sure. The end > of breve in Python? Not necessarily. Admittedly, this is largely a matter of taste. But of course, many seemingly critical decisions about programming are. Unfortunately, once I'd had my eyes opened to this limitation, it's now impossible for me not to notice. For example, when I see how much more natural Twisted's deferreds appear in MochiKit than in Python (where they originated), I can't help but believe that Twisted's failure to reach a larger audience is at least partially because that approach isn't as easily expressed in Python as it would be in a more expression-oriented language. In this case, it's not even that it can't be done (it clearly can), it's that the approach feels forced rather than natural. Overall, I'm left with the nagging suspicion that because of my language choice, I'm missing an important paradigm that would elevate my programming to the next level. In any case, I expect I'll stew in my current situation for a while longer at least. The unfortunate fact remains that most of the languages that speak to me (e.g. Boo and Io) don't have the broad range of libraries and frameworks available that Python does and ultimately this tends to outweigh my esoteric desires. Regards, Cliff From cliff at develix.com Thu Sep 11 19:40:29 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 10:40:29 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <20080911104433.0406fcdc@bhuda.mired.org> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> Message-ID: <1221154829.12242.413.camel@portable-evil> On Thu, 2008-09-11 at 10:44 -0400, Mike Meyer wrote: > On Wed, 10 Sep 2008 15:39:49 -0700 > Cliff Wells wrote: > > I strongly disagree. The artificial distinction between statements and > > expressions is the definition of *inconsistent*. Why do we have two > > versions of the "if" conditional? Why do we have "for" loops *and* list > > comprehensions? They express the same ideas, but the limitations in one > > required growing the language another direction. > > Because they are each more readable for the cases they handle. I > initially didn't like list comprehensions. In practice, they're not > very hard to write, and even easier to read. Doing away with for loops > because we have them would force you to construct - and then > immediately throw away - lists in the case where the for loop was > being executed for side effects. > > There have been a number of similar proposals - I've even generated > some. The general problem is that the results of embedding statements > with important indentation into expressions is ugly. While I've seen a > lot of words on this topic, I've seen very few concrete examples - and > those seem to come as often from opponents to the idea as they do from > you. > > So, how about some concrete examples? Show us translations of the > various if statements and for loops, so we have something concrete to > use to judge readability? Several examples have already been posted in this thread (which has probably gone on long enough). Anyway, as was mentioned earlier, Logix very much resembles what I'm describing: http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html#3.1 Regards, Cliff From josiah.carlson at gmail.com Thu Sep 11 19:41:27 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Thu, 11 Sep 2008 10:41:27 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221123925.12242.406.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <1221095665.12242.183.camel@portable-evil> <1221107691.12242.289.camel@portable-evil> <1221123925.12242.406.camel@portable-evil> Message-ID: On Thu, Sep 11, 2008 at 2:05 AM, Cliff Wells wrote: > On Wed, 2008-09-10 at 22:47 -0700, Josiah Carlson wrote: >> On Wed, Sep 10, 2008 at 9:34 PM, Cliff Wells wrote: >> > On Wed, 2008-09-10 at 18:30 -0700, Josiah Carlson wrote: >> >> On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells wrote: >> As a data point; everyone that I've taught Python over the years >> (undergrads and interns who only ever used Java before to 30-year >> programming veterans of apl/cobol/fortran), not a single one of them >> has ever had a problem with the difference between statements and >> expressions. It is possible that I've only ever worked with gifted >> students and coworkers, though this is unlikely. > > Without any supporting data, my hunch is that most people who migrate to > Python come from other imperative languages (myself included) and so > don't find the distinction foreign. Your hunch is likely correct. I personally came from Scheme and C/C++. Many of Python's functional constructs fit the part of me that got scheme, and everything else fit the part of me that got C/C++. >> What I've found (historically) is that any time I find myself trying >> to do something that is not available within the syntax of Python, >> it's usually because it's a bad idea. In your case, you want (for >> example) the following to be valid Python... >> >> lfcn = lambda x: ( >> if x: >> True >> else: >> False) >> >> Similarly... >> >> lfcn = lambda x: ( >> for i in xrange(x): >> yield fcn(i, x, ofcn(i)) >> ) >> >> Why? Initially it is for a templating language that you have designed >> that makes these kinds of things difficult to do during an automatic >> translation from working Python source code into compiled source code. >> Presumably something of the form... >> >> html [ body [ a (href='http://python.org') ["Python!"] ] ] >> >> To be converted into something like... >> >> def generate_html(out): >> out.write(''' >> >>     Python! >> >> ''') > > Actually, it's much simpler than that. The HTML tags are objects which > simply get serialized into their text representation. I specifically do > not generate Python source. In fact, you can simply do something like > this in the interactive interpreter: > >>>> print html [ body [ a (href='http://python.org') ["Python!"] ] ] >>>> > Python! > > This is more or less how Nevow Stan does it as well. Not translating to Python source code is a lot less fun ;), and also makes your templates far less dynamic in real-world scenarios. [snip] > Overall, I'm left with the nagging suspicion that because of my language > choice, I'm missing an important paradigm that would elevate my > programming to the next level. Then what the hell are you waiting around here for? If you are feeling constrained by Python (as a general-purpose language), try some others! You may want to add Haskell, Caml, or even Lua to your list of languages to check out. Maybe your Python will get better, or maybe you'll move on from Python. Who knows? > In any case, I expect I'll stew in my current situation for a while > longer at least. The unfortunate fact remains that most of the > languages that speak to me (e.g. Boo and Io) don't have the broad range > of libraries and frameworks available that Python does and ultimately > this tends to outweigh my esoteric desires. One thing that you need to remember about Python is that it's a general language. It's syntax and semantics are such that it works in a fairly large variety of situations. Because it's a general language, it sometimes doesn't do the things that would be convenient in a domain specific language. In particular, it doesn't have a lot of those things that would make functional programming more convenient (a shorter way of spelling 'lambda', everything is an expression, etc.), but no one language can be perfect for everyone. I wish you luck in your adventures with alternate languages. - Josiah From cliff at develix.com Thu Sep 11 20:08:35 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 11:08:35 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <20080911110113.1e64a8ef@bhuda.mired.org> References: <1221072180.12242.20.camel@portable-evil> <1221083684.12242.75.camel@portable-evil> <20080911110113.1e64a8ef@bhuda.mired.org> Message-ID: <1221156515.12242.426.camel@portable-evil> On Thu, 2008-09-11 at 11:01 -0400, Mike Meyer wrote: > On Wed, 10 Sep 2008 14:54:44 -0700 > Cliff Wells wrote: > > There were several design decisions to be made when designing Breve, the > > first and foremost being "will Breve templates be full-blown Python > > programs or will they be limited to expressions?". I chose the latter > > for a few reasons: > > That seems to be a very poor decision. If you want Breve templates to > play well with Python code (i.e. - be able to have Breve classes > inherit from Python classes and vice versa - but I guess without > statements, you don't have classes, so all of that is moot. Of course, > not having classes seems like a bad idea as well. Just to be clear because it appears to me you aren't quite following: Breve templates *are* Python code, or more exactly, they are a single Python expression. There is no preprocessor or interpreter that turns Breve into Python. It's just a bunch of class-magic. The decision to be a single large expression is largely because this is a *template* engine and an expression maps most directly to XML because XML is structured similarly: html [ head [ title [ 'A Brev? Template' ] ], body [ h1 [ 'Briefly, Brev?' ], br, div ( style = 'text-align: center;' ) [ span [ ''' As you can see, Brev? maps very directly to the final HTML output. ''' ] ] ] ] This is opposed to allowing the full of Python which would require a lot of syntax that is extraneous to the markup being generated (and most importantly would have caused a visual disjoint between the Breve source template and the final output). > Further, why do you feel that Breve templates - even if limited to > expressions - need to be limited to what Python expressions can do? Again, because Breve *is* Python. It's an internal DSL. Please see http://martinfowler.com/bliki/DomainSpecificLanguage.html for a definition of what that means. Another example of an internal DSL that's more widely used would be SQLAlchemy. Regards, Cliff From cliff at develix.com Thu Sep 11 20:19:24 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 11:19:24 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <20080911134818.45d0ea38@bhuda.mired.org> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> Message-ID: <1221157164.12242.437.camel@portable-evil> On Thu, 2008-09-11 at 13:48 -0400, Mike Meyer wrote: > On Thu, 11 Sep 2008 10:40:29 -0700 > Cliff Wells wrote: > > Several examples have already been posted in this thread (which has > > probably gone on long enough). > > Yes, I mentioned them. Roughly half have come from *other* people, so > they don't count. Since they were more or less exactly what I had in mind, I think they suffice. > If you really believe the thread has gone on long enough, then the > proposal is dead. If you are actually interested in seeing it happen, > then the real obstacle is that expressions built out of statements > that use indentation for control flow are *ugly*. That's only for people not used to seeing them or programming in that style. Proponents of functional and expression-oriented languages would strongly disagree. Whether or not they might be ugly *in Python* is a seemingly separate question, but I don't think it is. In fact, it seems it would be much more attractive than many traditional FP languages. > If you have a way > around it, I'd love to see it. If you don't - then again, the proposal > is dead. > > > Anyway, as was mentioned earlier, Logix very much resembles what I'm > > describing: > > > > http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html#3.1 > > That link doesn't have any examples, it just repeats many of the words > that have already been posted. Yes, the Logix documentation is sorely lacking (I believe the project to be long-dead), but the intent seemed clear enough to me. > If you have a link with real live > examples - in particular, showing how you can use for loops to replace > lcs and similar things, please provide it. >From an earlier example I posted: dispatch = { '1': lambda x: ( for i in range(x): if not x % 2: yield 0 else: yield 1 ), '2': lambda x: ( for i in range(x): yield i ) } for i in dispatch[val](1): print i Overall, if you are familiar with functional programming, it doesn't take a lot to imagine more examples. If you aren't, then it probably won't make much sense or seem too appealing in any case. Regards, Cliff From cliff at develix.com Thu Sep 11 20:41:11 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 11:41:11 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <1221095665.12242.183.camel@portable-evil> <1221107691.12242.289.camel@portable-evil> <1221123925.12242.406.camel@portable-evil> Message-ID: <1221158471.12242.452.camel@portable-evil> On Thu, 2008-09-11 at 10:41 -0700, Josiah Carlson wrote: > On Thu, Sep 11, 2008 at 2:05 AM, Cliff Wells wrote: > > Overall, I'm left with the nagging suspicion that because of my language > > choice, I'm missing an important paradigm that would elevate my > > programming to the next level. > > Then what the hell are you waiting around here for? If you are > feeling constrained by Python (as a general-purpose language), try > some others! You may want to add Haskell, Caml, or even Lua to your > list of languages to check out. Maybe your Python will get better, or > maybe you'll move on from Python. Who knows? Hey, I said I was lazy! ;-) Anyway, my biggest hurdle to most of the languages I've considered boils down to: 1) Lua - not expression based 2) Haskell - too ugly 3) Io - no libraries 4) Lisp - I find it borderline unreadable, plus I'd have to talk to other Lispers 5) Boo - I'd have to learn .NET and deal with the fact that 95% of the community is on Windows 6) Erlang - too ugly, too fast 7) Ruby - Too ugly, too slow, plus I'd probably have to make a cutesy website with cartoon animals and robots to show my fandom Two other options: 1) Logix - I could attempt to revive it. This is actually fairly appealing if a bit daunting to dive into. 2) Fork Python - I've considered making a proof-of-concept with a Python fork that does what I want, but it would be disappointing to discard it when it was inevitably rejected, and I have no intention of becoming a language maintainer. I think I'm going to play with Io for a bit in spite its shortcomings as it is explicitly a multiparadigm language and seems to look the cleanest. > > In any case, I expect I'll stew in my current situation for a while > > longer at least. The unfortunate fact remains that most of the > > languages that speak to me (e.g. Boo and Io) don't have the broad range > > of libraries and frameworks available that Python does and ultimately > > this tends to outweigh my esoteric desires. > > > One thing that you need to remember about Python is that it's a > general language. It's syntax and semantics are such that it works in > a fairly large variety of situations. Because it's a general > language, it sometimes doesn't do the things that would be convenient > in a domain specific language. In particular, it doesn't have a lot > of those things that would make functional programming more convenient > (a shorter way of spelling 'lambda', everything is an expression, > etc.), but no one language can be perfect for everyone. But it's so damn close =) > I wish you luck in your adventures with alternate languages. Thanks for all your time and thoughts. Regards, Cliff From cliff at develix.com Thu Sep 11 21:45:06 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 12:45:06 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <20080911145615.28de775f@bhuda.mired.org> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> Message-ID: <1221162306.12242.495.camel@portable-evil> On Thu, 2008-09-11 at 14:56 -0400, Mike Meyer wrote: > Here's where we disagree. I love functional programming, having > written 10s of thousands of lines of LISP in the past. The lack of the > ability to pass around functions in Ruby is why I haven't ever written > any. You can: def a(x,y) x + y end def x(b) send(b,3,4) end puts x(:a) # 7 The semantics are a bit different, but the result is the same (caveat: I'm no Ruby expert by any stretch of the imagination). Anyway, not important =) > And I consider the results of trying to cram python statements > into expressions as almost universally ugly. I'm willing to consider > you might have a solution, but you refuse to *show* it to me. I'm not sure what else you'd like to see. I've provided an example with a for-loop as generator and if-expressions. I've no intention of writing a full PEP covering all use-cases at this point. > > Overall, if you are familiar with functional programming, it doesn't > > take a lot to imagine more examples. If you aren't, then it probably > > won't make much sense or seem too appealing in any case. > > You're right - it's easy to image in more examples. Really, really > ugly ones, like: > > a = ( > if x < 23: > 18 > else: > 24 > ) + ( > if y < 23: > 29 > else: > 81 > ) > > The thought of running into something like the above in production > code is enough to make me bounce pretty much any proposal that would > allow it. Compared to that, > > a = (18 if x < 23 else 24) + (29 if y < 23 else 81) a = ( if x < 23: 18 else: 24 ) + ( if y < 23: 29 else: 81 ) Granted, today this is not valid Python, but supporting it doesn't seem to break existing Python code either. This is precisely how Logix does it (from the page I linked you to earlier): x = if 2+2==4: "I thought so" else: "hmmm..." In any case, your above code seems intentionally formatted to be unreadable, something that could be achieved under any circumstances. For example, here's your code reformatted in a fashion similar to your first example: a = ( 18 if x < 23 else 24 ) + ( 29 if y < 23 else 81 ) Just as unreadable (although really I don't find it terrible to comprehend... certainly no worse than a complicated listcomp). At the end of the day, no programming language is going to be able to enforce sane coding style (even Python's whitespace can be circumvented to a large degree). What we do gain with the idea I've forwarded is a single, more flexible if-expression in place of an if-statement and an if-operator with inverted syntax. Regards, Cliff From cliff at develix.com Thu Sep 11 23:06:58 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 14:06:58 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <20080911162143.2101a624@bhuda.mired.org> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> Message-ID: <1221167218.12242.524.camel@portable-evil> On Thu, 2008-09-11 at 16:21 -0400, Mike Meyer wrote: > On Thu, 11 Sep 2008 12:45:06 -0700 > Cliff Wells wrote: > > > The thought of running into something like the above in production > > > code is enough to make me bounce pretty much any proposal that would > > > allow it. Compared to that, > > > > > > a = (18 if x < 23 else 24) + (29 if y < 23 else 81) > > > > a = ( if x < 23: 18 else: 24 ) + ( if y < 23: 29 else: 81 ) > > And you ignored the constraint that you had to leave the newlines and > indentations in the statement. Sorry, I'm not going to satisfy "now do it with one hand behind your back" sort of challenges. This is the syntax I'm proposing, so why would I provide it any other way? > Also, you've claimed a number of times > that you didn't want to change the syntax of python, but this clearly > requires a syntax change, as current python syntax doesn't allow > multiple colon-separated statements on a line. No, I stated I didn't want to *add* any additional syntax (i.e. block delimiters, keywords or whatnot), nor do I want to break existing code. I certainly propose changing the grammar (albeit in a demonstrably backwards-compatible way). > > Granted, today this is not valid Python, but supporting it doesn't seem > > to break existing Python code either. This is precisely how Logix does > > it (from the page I linked you to earlier): > > > > x = if 2+2==4: "I thought so" else: "hmmm..." > > > > In any case, your above code seems intentionally formatted to be > > unreadable, something that could be achieved under any circumstances. > > For example, here's your code reformatted in a fashion similar to your > > first example: > > Yup, and if that was what someone was proposing, it'd be a reason to > reject it. But nobody is proposing turning expressions into > statements, which is what you just did. No, I turned statements into expressions, which is what I've been proposing since the very first post. I cannot possibly fathom where you came up with the exact opposite. You continue to be stuck in a rut of demanding that something be a statement when this is simply not a requirement. Perfectly legal (although useless) Python: 1 2 3 Note that this does not somehow magically turn numbers into statements. It affirms that expressions can be used as statements (something that is already true in Python). Ask yourself, is a function call a statement or an expression? It's an expression (a function call *always* returns a value), and yet it can be used alone: def foo(x): return x**2 foo(2) # expression used as a statement y = foo(3) # expression used as an expression > > What we do gain with the idea I've forwarded is a single, more flexible > > if-expression in place of an if-statement and an if-operator with > > inverted syntax. > > But that's not what you've been claiming you wanted. You've been > claiming you wanted statements to be expressions. Uh... yeah. Seriously: which part of "this is an example of turning a particular Python statement into an expression" am I not conveying to you? > I claim that doing > that makes for ugly code, because statements - with newlines and > whitespace as a critical part of the syntax - are ugly to embed in > expressions. I've asked for examples and not gotten them. I provide an > ugly example of statements embedded in an expressions, and you respond > by providing an alternative trinary operator to rewrite it. I did not. I'm going to let you try to understand that one line of code better after you've realized you wouldn't need an operator if the if-statement were an if-expression. > I still want to see how you propose to embed *statements* into > arbitrary expressions, complete with newlines and proper > indentation. Say... > a = ( > if x < 24: > go_make_some_side_effects() > a_nice_long_function() * another_nice_long_function() > else: > go_make_different_side_effects() > a_twisty_little_function() * a_nice_long_function() > ) \ > + ( > if y < 23: > go_make_some_different_side_effects() > a_twisty_little_function() * another_nice_long_function() > else: > go_make_different_side_effects() > a_nice_long_function() * a_twisty_little_function() > ) > > This is still simply bletcherous compared to the obvious legal > rewrite. Ignoring your superfluous backslash, the above is a perfectly valid example. Of course, like all contrived examples, it doesn't necessarily convey *why* you'd want to do this, but the syntax is correct. Also, go_make_some_side_effects() is probably ill-advised, and not something usually done in an FP style which you claim to be familiar with. Imperative programming is all about side-effects whereas functional programming is all about avoiding them. > But it's about the cleanest way I've seen to deal with the > issue. Using operator.add might be a bit cleaner, but not enough so to > justify using operator.add. > > And we haven't gotten *really* warped yet, with statements appearing > as controlling expressions inside of other statements. While making > this "simple" changes allows you to make a few use cases simpler, it > also makes some truly ugly things seem reasonable. Frankly, I'd rather > let the experts deal with those use cases individually with > well-thought out extensions than open this can of worms - unless you > can show how to do those kinds of embeddings in ways that *aren't* > ugly. I point you to the entire history of FP programming and we can leave it at that. If you've programmed in Lisp as you claim then I cannot see any reason to rehash what you, by definition, must already know, but keep pretending not to. Unless you're willing to admit you simply don't know FP or that you are being intentionally obtuse for some unfathomable reason, then I'm not interested in pursuing this discussion with you any further. If it's a matter of me not being clear, then I apologize, but frankly I've been as clear as I am able so further discussion would still certainly seem to be fruitless. Regards, Cliff From rhamph at gmail.com Thu Sep 11 23:40:35 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 11 Sep 2008 15:40:35 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221167218.12242.524.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> Message-ID: On Thu, Sep 11, 2008 at 3:06 PM, Cliff Wells wrote: > On Thu, 2008-09-11 at 16:21 -0400, Mike Meyer wrote: >> Also, you've claimed a number of times >> that you didn't want to change the syntax of python, but this clearly >> requires a syntax change, as current python syntax doesn't allow >> multiple colon-separated statements on a line. > > No, I stated I didn't want to *add* any additional syntax (i.e. block > delimiters, keywords or whatnot), nor do I want to break existing code. > I certainly propose changing the grammar (albeit in a demonstrably > backwards-compatible way). We keep disagreeing on this point. You're not adding any new tokens, but you are adding a ton of new syntax! Not to mention that doing it generically doesn't work (a for-loop needs to evaluate eagerly, but mustn't retain all the values it contained), and your primary example (dispatch-dict) only needs lambda to allow statements, rather than the broad changes you suggest. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Thu Sep 11 23:53:07 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 11 Sep 2008 15:53:07 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221167218.12242.524.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> Message-ID: On Thu, Sep 11, 2008 at 3:06 PM, Cliff Wells wrote: > Also, > go_make_some_side_effects() is probably ill-advised, and not something > usually done in an FP style which you claim to be familiar with. > Imperative programming is all about side-effects whereas functional > programming is all about avoiding them. This isn't true when applied to python. We're about halfway in between, regularly avoiding side effects (immutable int/str, sorted(), iteration is generic and only covers reading), while happily doing side-effects when appropriate. Moreover, although iteration is built on mutation (the iterator object's state changes as you go through it), good style is to contain that in a single function. The end result is often directly translatable into a side-effect-free language, only our version is easier to read; all they offer is a guarantee of no side effects. -- Adam Olsen, aka Rhamphoryncus From bruce at leapyear.org Fri Sep 12 00:14:40 2008 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 11 Sep 2008 15:14:40 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221167218.12242.524.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> Message-ID: I agree that making every statement also an expression has merits, as does FP. However, you're handwaving about how this fits into python. The scope of an if is defined syntactically by indentation. Your proposal changes that in some way that I can't quite tell. Is it that you can put a single expression between the if and the else? Or something else? When would line breaks be optional where they are now required? Simply saying "make everything an expression" is insufficient. Saying "it's just lifting a restriction" is also insufficient. In many languages there are restrictions on using the + operators with non-numbers. Why can't we just remove that restriction? Well, we have to define the precise semantics in order to do that, e.g., what does []+{} mean? You said that the value of a for loop would have the value [] if it didn't include yield, i.e., for x in y: foo(x) evaluates to [] while you said if x: foo(x) else: foo(0) evaluates to foo(something). That seems inconsistent. Is that inconsistency a good thing? For that matter, why return a list rather than () or an empty generator? [[As an aside: why can't I write {foo(x) for x in y} which is the equivalent of doing d[x] = foo(x) for all x. I've just written the equivalent loop several times recently.]] To answer these questions, the right thing to do is to write a list of exactly what new syntax and semantics you are proposing. I'm not saying that I (or anyone else) will agree with what you propose but I certainly can't agree with the vague proposal you've made so far. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Fri Sep 12 02:24:03 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 11 Sep 2008 20:24:03 -0400 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221157164.12242.437.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081516.12242.53.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> Message-ID: Cliff Wells wrote: >> If you have a link with real live >> examples - in particular, showing how you can use for loops to replace >> lcs and similar things, please provide it. > >>From an earlier example I posted: > > dispatch = { > '1': lambda x: ( > for i in range(x): > if not x % 2: > yield 0 > else: > yield 1 > ), > > '2': lambda x: ( > for i in range(x): > yield i > ) > } > > for i in dispatch[val](1): > print i To me, this is the most (perhaps only ;-) readable such example. But it is unnecessary. Even without metaclasses, as someone else suggested (and thank you for the idea), the following works today, and is even more readable to me. class disclass(): # 3.0 def f1(x): for i in range(x): if not x % 2: yield 0 else: yield 1 def f2(x): for i in range(x): yield i dispatch = disclass.__dict__ val = '1' for i in dispatch['f'+val](2): print(i) for i in dispatch['f'+val](3): print(i) val = '2' for i in dispatch['f'+val](4): print(i) >>> 0 0 1 1 1 0 1 2 3 In 3.0, a class decorator could, I believe, both replace the class with its dict, and strip the leading 'f' from the 'fn' keys. So there is another 'need' for generalized lambda that isn't. Terry Jan Reedy From cliff at develix.com Fri Sep 12 03:01:55 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 18:01:55 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> Message-ID: <1221181315.12242.552.camel@portable-evil> On Thu, 2008-09-11 at 15:14 -0700, Bruce Leban wrote: > I agree that making every statement also an expression has merits, as > does FP. However, you're handwaving about how this fits into python. Sure. I intentionally handwave for two reasons: 1) it's a bit of a waste of time to detail something like this when it's universally rejected 2) I didn't want to bicker over details. I wanted to forward the concept. > The scope of an if is defined syntactically by indentation. Your > proposal changes that in some way that I can't quite tell. Is it that > you can put a single expression between the if and the else? Or > something else? When would line breaks be optional where they are now > required? I'd say anytime it's ambiguous (i.e. much like you would for parenthesizing any other group of expressions). Logix manages some of the ambiguity by defining ';' as an operator that simply evaluates to the rightmost value. I think this makes a lot of sense. if x: y else: z # seems obvious if x: y; y+1 else: z; z+1 # evaluates to y+1 or z+1 depending on x and is equivalent to if x: y y+1 else: z z+1 A the primary question that arises is what to make of assignment. GvR is notoriously against allowing constructs such as: if ( x=1 ): y and I think his reasoning is sound (this is usually a programming error). I tend to think it should allowed (for consistency), but it could be defined as a syntax error in this context. At any rate, I'll resort to the same argument as is used against people who demand static type declarations: unit testing =). Frankly, either is acceptable to me. One thing that would have to change about assignment is it would definitely *have* to be an expression (and evaluate to the assigned value). Whether it's allowed in any particular context is open. > Simply saying "make everything an expression" is insufficient. Saying > "it's just lifting a restriction" is also insufficient. In many > languages there are restrictions on using the + operators with > non-numbers. Why can't we just remove that restriction? Well, we have > to define the precise semantics in order to do that, e.g., what does > []+{} mean? > > > You said that the value of a for loop would have the value [] if it > didn't include yield, i.e., > for x in y: foo(x) > > evaluates to [] while you said > if x: foo(x) else: foo(0) > evaluates to foo(something). That seems inconsistent. > Is that inconsistency a good thing? For that matter, why return a list > rather than () or an empty generator? [[As an aside: why can't I write > {foo(x) for x in y} which is the equivalent of doing d[x] = foo(x) for > all x. I've just written the equivalent loop several times recently.]] Both evaluate to *something*, just that the non-yielding for-loop evaluates to an empty list (probably a good indication it's not being used in an enclosing expression). It could probably evaluate to None, if that seemed more reasonable. I tend to prefer an empty list as it allows for looping without testing for an iterable. Incidentally, how would an empty generator differ from []? I'll concede it might make more logical sense, but would it differ in application? > To answer these questions, the right thing to do is to write a list of > exactly what new syntax and semantics you are proposing. I'm not > saying that I (or anyone else) will agree with what you propose but I > certainly can't agree with the vague proposal you've made so far. I'd probably defer heavily to an existing implementation that does exactly what I'm referring to (except I'm not espousing macros or other metaprogramming facilities Logix provides): http://www.livelogix.net/logix/index.html Whether or not this is the ideal situation is arguable, but those arguments would need to be much further down the road. Right now there's not even agreement on whether the overarching concept is even appropriate for Python. Cliff From grosser.meister.morti at gmx.net Fri Sep 12 03:15:23 2008 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Fri, 12 Sep 2008 03:15:23 +0200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221072180.12242.20.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> Message-ID: <48C9C2AB.5030103@gmx.net> The only thing I'd like would be anonymous classes (like in java) and anonymous functions: class A(object): def some_method(self): pass def function_that_wants_a_A(a): ... function_that_wants_a_A(A(): def some_method(self): # I really like this feature of java # granted, it makes a lot more sense in the # context of a language like java. # there it's very handy print "this is my derived anonymous class" ) def function_that_wants_a_callback(callback): ... function_that_wants_a_callback(def(a,b,c): # do a lot of stuff that does not fit into a lambda # this reminds a bit of rubies closures ;) ) I don't think anything else concerning statement as expression is in any way necessary or handy. From cliff at develix.com Fri Sep 12 03:16:41 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 18:16:41 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> Message-ID: <1221182201.12242.568.camel@portable-evil> On Thu, 2008-09-11 at 15:53 -0600, Adam Olsen wrote: > On Thu, Sep 11, 2008 at 3:06 PM, Cliff Wells wrote: > > Also, > > go_make_some_side_effects() is probably ill-advised, and not something > > usually done in an FP style which you claim to be familiar with. > > Imperative programming is all about side-effects whereas functional > > programming is all about avoiding them. > > This isn't true when applied to python. We're about halfway in > between, regularly avoiding side effects (immutable int/str, sorted(), > iteration is generic and only covers reading), while happily doing > side-effects when appropriate. Well, I'd argue that *any* statement is causing a side-effect as it affects the flow of control or values of objects outside its own scope. Even the following represents one side-effect (defining a class in the outer scope): class Foo(object): pass Expressions on the other hand, *evaluate* into the outer scope. They must be explicitly assigned to have any long-term effect once they've completed. One way to look at it is that statements "push" values into the enclosing scope whereas expressions "offer" values to it. I'd call the "pushing" a side-effect. Note that I am not claiming side-effects to be universally bad things (they often are, but they are just as often unavoidable to do real work, i.e. printing to the terminal). > Moreover, although iteration is built on mutation (the iterator > object's state changes as you go through it), good style is to contain > that in a single function. The end result is often directly > translatable into a side-effect-free language, only our version is > easier to read; all they offer is a guarantee of no side effects. Not 100% I'm sure I follow this, but I think I might agree ;-) Cliff From cliff at develix.com Fri Sep 12 03:19:31 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 18:19:31 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48C9C2AB.5030103@gmx.net> References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> Message-ID: <1221182371.12242.570.camel@portable-evil> On Fri, 2008-09-12 at 00:10 +0200, Mathias Panzenb?ck wrote: > The only thing I'd like would be anonymous classes (like in java) and anonymous > functions: > I don't think anything else concerning statement as expression is in any way > necessary or handy. At one time I also perceived a shortcoming in lambda. You could continue to see it this way, or you could see it (like I now do) that it's not lambda that's limited (it effectively allows an arbitrary number of expressions), but rather the fact that a statement cannot be be used as an expression. For instance, the following is currently legal in Python: lambda x, y: ( x + y, x**2, y**2 ) But this is not: lambda x, y: ( if x > y: x else: y ) Whether or not it's illegal is due to lambda being limited or because statements are castrated expressions is all a matter of your point of view. Extending lambda to allow statements would certainly fix *lambda*, but making statements into expressions fixes Python (IMHO, of course). Regards, Cliff From guido at python.org Fri Sep 12 03:38:39 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 11 Sep 2008 18:38:39 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221182371.12242.570.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> Message-ID: Hey Cliff, I think the only way to move forward in this discussion is if you could write a complete parser for the variant of Python that you'd like to see. I expect that you'll say "but I don't know enough about parser technology to do so" -- but then my response is "then you don't know to understand the difficulty of the task, so shut up." If you want to continue this discussion you'll have to prove that it is possible. That in turn will force you to think about many of the details that you're currently brushing away as irrelevant -- and you'll find that they are actually highly relevant. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rhamph at gmail.com Fri Sep 12 05:34:33 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 11 Sep 2008 21:34:33 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221182201.12242.568.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221182201.12242.568.camel@portable-evil> Message-ID: On Thu, Sep 11, 2008 at 7:16 PM, Cliff Wells wrote: > On Thu, 2008-09-11 at 15:53 -0600, Adam Olsen wrote: >> On Thu, Sep 11, 2008 at 3:06 PM, Cliff Wells wrote: >> > Also, >> > go_make_some_side_effects() is probably ill-advised, and not something >> > usually done in an FP style which you claim to be familiar with. >> > Imperative programming is all about side-effects whereas functional >> > programming is all about avoiding them. >> >> This isn't true when applied to python. We're about halfway in >> between, regularly avoiding side effects (immutable int/str, sorted(), >> iteration is generic and only covers reading), while happily doing >> side-effects when appropriate. > > Well, I'd argue that *any* statement is causing a side-effect as it > affects the flow of control or values of objects outside its own scope. In a strict technical sense, sure. Perhaps it's better to say that python encourages *containing* side-effects. -- Adam Olsen, aka Rhamphoryncus From cliff at develix.com Fri Sep 12 06:08:28 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 21:08:28 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> Message-ID: <1221192508.12242.616.camel@portable-evil> On Thu, 2008-09-11 at 18:38 -0700, Guido van Rossum wrote: > Hey Cliff, > > I think the only way to move forward in this discussion is if you > could write a complete parser for the variant of Python that you'd > like to see. As I've pointed out, this has already been done (although the project appears dead): http://www.livelogix.net/logix/index.html http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html Logix is actually much more than a Python interpreter, but that's tangential to this discussion. Logix also conveniently runs on the Python VM. It also removes the distinction between statements and expressions and is nearly code-compatible with Python (it has two significant shortcomings and two minor ones, none of which have to do with the topic at hand). http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html > I expect that you'll say "but I don't know enough about parser > technology to do so" -- but then my response is "then you don't know > to understand the difficulty of the task, so shut up." > If you want to continue this discussion you'll have to > prove that it is possible. Or I can continue to say what I like and you can choose to ignore it. I obviously believe it's a worthwhile discussion and certainly enjoy the debate, but anyone who doesn't is best served by not joining it. I don't recall bringing this to python-dev as a serious proposal nor submitting it as a PEP, so I think an informal exchange of ideas without spending six months on a functional prototype is perfectly acceptable. That aside, it's already been proven that the syntax and concept is plausible in Logix. Whether or not it could be implemented in CPython is another matter, and whether or not the result would be accepted is yet a another matter. > That in turn will force you to think about many of the details that > you're currently brushing away as irrelevant -- and you'll find that > they are actually highly relevant. I'm not brushing them away as irrelevant, I'm setting them aside until it's decided that it's even worth pursuing. I don't know about you, but I don't bother arguing with my wife about the color of a new cars until we're certain we need to buy one. I certainly agree that if I decided to move this beyond idle discussion that a prototype would be a critical aspect and I fully expect there would be many issues to be overcome. I don't believe such a change would be simple, but neither do I believe it's insurmountable. Regards, Cliff From guido at python.org Fri Sep 12 06:36:40 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 11 Sep 2008 21:36:40 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221192508.12242.616.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> <1221192508.12242.616.camel@portable-evil> Message-ID: On Thu, Sep 11, 2008 at 9:08 PM, Cliff Wells wrote: > On Thu, 2008-09-11 at 18:38 -0700, Guido van Rossum wrote: >> Hey Cliff, >> >> I think the only way to move forward in this discussion is if you >> could write a complete parser for the variant of Python that you'd >> like to see. > > As I've pointed out, this has already been done (although the project > appears dead): > > http://www.livelogix.net/logix/index.html > > http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html > > Logix is actually much more than a Python interpreter, but that's > tangential to this discussion. Logix also conveniently runs on the > Python VM. It also removes the distinction between statements and > expressions and is nearly code-compatible with Python (it has two > significant shortcomings and two minor ones, none of which have to do > with the topic at hand). > > http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html > >> I expect that you'll say "but I don't know enough about parser >> technology to do so" -- but then my response is "then you don't know >> to understand the difficulty of the task, so shut up." >> If you want to continue this discussion you'll have to >> prove that it is possible. > > Or I can continue to say what I like and you can choose to ignore it. I > obviously believe it's a worthwhile discussion and certainly enjoy the > debate, but anyone who doesn't is best served by not joining it. I > don't recall bringing this to python-dev as a serious proposal nor > submitting it as a PEP, so I think an informal exchange of ideas without > spending six months on a functional prototype is perfectly acceptable. > > That aside, it's already been proven that the syntax and concept is > plausible in Logix. Whether or not it could be implemented in CPython > is another matter, and whether or not the result would be accepted is > yet a another matter. > >> That in turn will force you to think about many of the details that >> you're currently brushing away as irrelevant -- and you'll find that >> they are actually highly relevant. > > I'm not brushing them away as irrelevant, I'm setting them aside until > it's decided that it's even worth pursuing. I don't know about you, but > I don't bother arguing with my wife about the color of a new cars until > we're certain we need to buy one. > > I certainly agree that if I decided to move this beyond idle discussion > that a prototype would be a critical aspect and I fully expect there > would be many issues to be overcome. I don't believe such a change > would be simple, but neither do I believe it's insurmountable. Well, if you are asking for my personal opinion about the viability of this idea, I think it's not viable, but would like to be proven wrong (or right!) just so that this topic can be put to bed for good. I'm not swayed by the existence of Logix; it appears to be using a completely different way of interpreting whitespace, which seems incompatible with current Python, but they don't seem to give the exact rules, only some hints. Have you used it? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From cliff at develix.com Fri Sep 12 07:34:44 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 22:34:44 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> <1221192508.12242.616.camel@portable-evil> Message-ID: <1221197684.12242.645.camel@portable-evil> On Thu, 2008-09-11 at 21:36 -0700, Guido van Rossum wrote: > On Thu, Sep 11, 2008 at 9:08 PM, Cliff Wells wrote: > > I certainly agree that if I decided to move this beyond idle discussion > > that a prototype would be a critical aspect and I fully expect there > > would be many issues to be overcome. I don't believe such a change > > would be simple, but neither do I believe it's insurmountable. > > Well, if you are asking for my personal opinion about the viability of > this idea, I think it's not viable, but would like to be proven wrong > (or right!) just so that this topic can be put to bed for good. Well, that was actually my goal in bringing it up here: I've got something nagging me and the only way I'm going to put it to rest is to hash it out in a roomful of devil's advocates =) > I'm not swayed by the existence of Logix; it appears to be using a > completely different way of interpreting whitespace, which seems > incompatible with current Python, but they don't seem to give the > exact rules, only some hints. Have you used it? I've imported Logix into a Pylons session and used some of it's features without issue. It's quite probable that this may have only been successful because Pylons itself was precompiled with CPython and the only code affected was the particular module I was compiling at the time. I'm curious which aspect of Logix' whitespace handling you suspect to be incompatible (aside from the line-continuation stuff). In any case, as fascinating as Logix is, I find it mostly interesting (to this debate) as an example of what such a syntax could appear like. Nevertheless, I think I have a compromise: in lieu of writing an interpreter, I'll investigate and properly document Logix (something I've considered in the past anyway). This way we can at least have an implementation to argue over without me needing to quit my day job to make a point ;-) Regards, Cliff From cliff at develix.com Fri Sep 12 07:46:30 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 11 Sep 2008 22:46:30 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221197684.12242.645.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> <1221192508.12242.616.camel@portable-evil> <1221197684.12242.645.camel@portable-evil> Message-ID: <1221198390.12242.654.camel@portable-evil> On Thu, 2008-09-11 at 22:34 -0700, Cliff Wells wrote: > Nevertheless, I think I have a compromise: in lieu of writing an > interpreter, I'll investigate and properly document Logix (something > I've considered in the past anyway). This way we can at least have an > implementation to argue over without me needing to quit my day job to > make a point ;-) Bah, it appears it doesn't work under 2.5 (I haven't tried it since 2.4). I guess my work is cut out for me :P Regards, Cliff From scott+python-ideas at scottdial.com Fri Sep 12 08:33:42 2008 From: scott+python-ideas at scottdial.com (Scott Dial) Date: Fri, 12 Sep 2008 02:33:42 -0400 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221181315.12242.552.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> Message-ID: <48CA0D46.5010604@scottdial.com> Cliff Wells wrote: > Sure. I intentionally handwave for two reasons: > 1) it's a bit of a waste of time to detail something like this when it's > universally rejected And yet this thread, persists.. *sigh* -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From scott+python-ideas at scottdial.com Fri Sep 12 08:40:07 2008 From: scott+python-ideas at scottdial.com (Scott Dial) Date: Fri, 12 Sep 2008 02:40:07 -0400 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221192508.12242.616.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> <1221192508.12242.616.camel@portable-evil> Message-ID: <48CA0EC7.5070700@scottdial.com> Cliff Wells wrote: >> That in turn will force you to think about many of the details that >> you're currently brushing away as irrelevant -- and you'll find that >> they are actually highly relevant. > > I'm not brushing them away as irrelevant, I'm setting them aside until > it's decided that it's even worth pursuing. I don't know about you, but > I don't bother arguing with my wife about the color of a new cars until > we're certain we need to buy one. I'm sorry, but the points people are bringing up are not bikeshedding. They are very real points that are more analogous to whether the car has wheels or flies on pixie dust. I'm sorry, but you can't cast aside the subtleties of turning statements into expressions as bikeshedding arguments. Many of the people who gave you very genuine feedback pointed out particularly difficult to rectify problems (like how exactly do you notate a suite if you eliminate the required indent tokens?) and your response to them is that they are missing the point.. You want to change the language, the syntax of that change is *exactly* the point. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From cliff at develix.com Fri Sep 12 16:41:20 2008 From: cliff at develix.com (Cliff Wells) Date: Fri, 12 Sep 2008 07:41:20 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CA0EC7.5070700@scottdial.com> References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> <1221192508.12242.616.camel@portable-evil> <48CA0EC7.5070700@scottdial.com> Message-ID: <1221230480.12242.680.camel@portable-evil> On Fri, 2008-09-12 at 02:40 -0400, Scott Dial wrote: > Cliff Wells wrote: > >> That in turn will force you to think about many of the details that > >> you're currently brushing away as irrelevant -- and you'll find that > >> they are actually highly relevant. > > > > I'm not brushing them away as irrelevant, I'm setting them aside until > > it's decided that it's even worth pursuing. I don't know about you, but > > I don't bother arguing with my wife about the color of a new cars until > > we're certain we need to buy one. > > I'm sorry, but the points people are bringing up are not bikeshedding. > They are very real points that are more analogous to whether the car has > wheels or flies on pixie dust. Fair enough. I simply think there's a larger philosophical issue: is the potential for deeply nested code a greater concern than an ever-expanding suite of special-case constructs? That needs to be agreed on before deciding on a specific way of addressing it. Even if we agreed on the larger issue I've forwarded, what I'm suggesting may not be the only way of addressing it. > I'm sorry, but you can't cast aside the > subtleties of turning statements into expressions as bikeshedding > arguments. Many of the people who gave you very genuine feedback pointed > out particularly difficult to rectify problems (like how exactly do you > notate a suite if you eliminate the required indent tokens?) To be clear, I don't suggest eliminating the required indentation for multi-line blocks (although it seems you could intentionally dodge it through judicious use of the ';' operator and parentheses, much like you can with expressions and operators today). > and your > response to them is that they are missing the point.. You want to change > the language, the syntax of that change is *exactly* the point. Well, I'll take the blame for not spending more time separating out the fact that I'm presenting both a problem and a potential solution which has led to a conflation of the two to some degree. I think the solution I've presented helps demonstrate that there *is* a problem and so I find it difficult to present the problem in the solution's absence. As I mentioned in my first post, the current situation has arisen because of a philosophy ("flat is better than nested"). This seems a valid position, but I'm arguing that it has proven over time to have as many pitfalls (in terms of complexity) as the problem it seeks to avoid. Hopefully that makes it more clear why I'm trying to get people to see (and hopefully agree) that there's a problem before bickering over the details of its solution. Regards, Cliff From santagada at gmail.com Fri Sep 12 16:56:26 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Fri, 12 Sep 2008 11:56:26 -0300 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221230480.12242.680.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> <1221192508.12242.616.camel@portable-evil> <48CA0EC7.5070700@scottdial.com> <1221230480.12242.680.camel@portable-evil> Message-ID: <8F0685DA-AF1B-456B-845D-5B2731B8F8B2@gmail.com> On Sep 12, 2008, at 11:41 AM, Cliff Wells wrote: > Fair enough. I simply think there's a larger philosophical issue: is > the potential for deeply nested code a greater concern than an > ever-expanding suite of special-case constructs? That needs to be > agreed on before deciding on a specific way of addressing it. I love some of the constructs of python like list comprehension but yes the new "if" construct don't strike me as a nice adition. And maybe your solution is a good one, but I will have to see it working. If you want to alter pypy to do it you will probably have to improve the parser generator that is being used right now, but you can also try to do it on ironpython or jython. -------------- next part -------------- An HTML attachment was scrubbed... URL: From santagada at gmail.com Fri Sep 12 16:59:07 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Fri, 12 Sep 2008 11:59:07 -0300 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221230480.12242.680.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> <1221192508.12242.616.camel@portable-evil> <48CA0EC7.5070700@scottdial.com> <1221230480.12242.680.camel@portable-evil> Message-ID: <2980188B-0122-4F18-9D99-93C8655F05E9@gmail.com> On Sep 12, 2008, at 11:41 AM, Cliff Wells wrote: > Fair enough. I simply think there's a larger philosophical issue: is > the potential for deeply nested code a greater concern than an > ever-expanding suite of special-case constructs? That needs to be > agreed on before deciding on a specific way of addressing it. I love some of the constructs of python like list comprehension but yes the new "if" construct don't strike me as a nice adition. And maybe your solution is a good one, but I will have to see it working. If you want to alter pypy to do it you will probably have to improve the parser generator that is being used right now, but you can also try to do it on ironpython or jython. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cliff at develix.com Fri Sep 12 17:55:51 2008 From: cliff at develix.com (Cliff Wells) Date: Fri, 12 Sep 2008 08:55:51 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <20080912110014.18a1029d@bhuda.mired.org> References: <1221072180.12242.20.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> Message-ID: <1221234951.8492.40.camel@portable-evil> On Fri, 2008-09-12 at 11:00 -0400, Mike Meyer wrote: > On Thu, 11 Sep 2008 18:01:55 -0700 > Cliff Wells wrote: > > > On Thu, 2008-09-11 at 15:14 -0700, Bruce Leban wrote: > > > I agree that making every statement also an expression has merits, as > > > does FP. However, you're handwaving about how this fits into python. > > > > Sure. I intentionally handwave for two reasons: > > > > 1) it's a bit of a waste of time to detail something like this when it's > > universally rejected > > 2) I didn't want to bicker over details. I wanted to forward the > > concept. > > Ah, I get it. I thought you knew something about the Python user > community. Clearly, that's not the case. I'll concede I more-or-less stopped reading c.l.py at least 3 or 4 years ago ;-) > If you look through the > proposed changes to python, you'll find this kind of thing (changes > that introduce a single powerful feature that let you manipulate > statements in expressions) surfacing at regular intervals over at > least the past 15 years. The idea is almost never rejected > outright. In fact, if you look, that's what's happening here - people > aren't rejecting your position outright, but trying to get enough > information to figure out how it's going to affect the entire > language. Yes, I accept the responsibility for conflating what I see as a problem with a possible solution to that problem (the two were inextricably linked in my mind). > That's another place where the Python community differs from other > language communities. We don't believe in features for the sake of > features (whether you get them by adding or removing things); we > believe the features in the language should interact well with each > other. So while some feature may scratch a particular itch - say to do > FP programming in Python - the fact that it turns ugly when used in > non-FP code, or requires ignoring fundamental language layout rules, > are not things that tie one hand behind your back, but crucial > problems that must be solved if the feature is to be added. So even > though the idea behind a feature may be generally acceptable - and > that seems to be the case here - it won't be adopted until those > details get worked out. Agreed, and I had intended to move to that stage once I felt there was a consensus that there is actually something that needs solving. What I failed to make clear in my position is that I'm not trying to add a feature for a it's own sake, rather that the current process for limiting the growth of the Python language is doomed to failure. I'm going to try to rephrase the fundamental problem I see here and hopefully I won't fail so dismally this time: 1) I assert that small is better than large when it comes to limiting complexity. 2) The primary method of limiting Python's core feature growth is for the BDFL to dig in his heels and reject ideas until they are adequately shown to be broadly needed and implementable. 3) This method has failed in the past*. In fact, I assert that this method is guaranteed to fail unless all ideas that add syntactical structures are rejected outright, regardless of their utility. It cannot limit the growth of Python's core, it can only limit the rate at which it grows. 4) As Python moves into new domains, as new programming languages come into vogue, and as the art of programming itself advances (albeit glacially), users will perceive needs in Python and will clamor for extensions. Some of these will eventually be accepted. 5) Given the added desire to maintain backwards-compatibility, old features will not be shed as fast as new ones are added (unless that becomes part of the process, which doesn't seem practicable to me). 6) I believe that a large class of these features could be rendered moot or redundant if the language embraced a more sweeping and fundamental change. While this won't absolutely prevent the language's growth, it provides a built-in deterrent. * See the ternary if-operator for an example - steadily rejected by GvR for many years and then finally accepted, probably out of exhaustion. There's an old quote by Larry Wall: "Perl is ugly because people wanted it that way". Perl took the same approach to limiting features that Python does, with the notable difference that Larry didn't reject ideas as consistently as Guido (or apparently at all). Nevertheless, we're now simply discussing the *rate* at which the language becomes large and inconsistent rather than whether it will or not. If we agree that the issues I outline above are valid, then I think we can start bickering over possible solutions and how those solutions would affect Python on the whole. I apologize for my previous response to you. It was clearly my own failing to properly explain my position that led to our exchange. Regards, Cliff From josiah.carlson at gmail.com Fri Sep 12 18:51:46 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Fri, 12 Sep 2008 09:51:46 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221234951.8492.40.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> Message-ID: On Fri, Sep 12, 2008 at 8:55 AM, Cliff Wells wrote: > On Fri, 2008-09-12 at 11:00 -0400, Mike Meyer wrote: >> On Thu, 11 Sep 2008 18:01:55 -0700 >> Cliff Wells wrote: >> >> > On Thu, 2008-09-11 at 15:14 -0700, Bruce Leban wrote: >> > > I agree that making every statement also an expression has merits, as >> > > does FP. However, you're handwaving about how this fits into python. >> > >> > Sure. I intentionally handwave for two reasons: >> > >> > 1) it's a bit of a waste of time to detail something like this when it's >> > universally rejected >> > 2) I didn't want to bicker over details. I wanted to forward the >> > concept. >> >> Ah, I get it. I thought you knew something about the Python user >> community. Clearly, that's not the case. > > I'll concede I more-or-less stopped reading c.l.py at least 3 or 4 years > ago ;-) > >> If you look through the >> proposed changes to python, you'll find this kind of thing (changes >> that introduce a single powerful feature that let you manipulate >> statements in expressions) surfacing at regular intervals over at >> least the past 15 years. The idea is almost never rejected >> outright. In fact, if you look, that's what's happening here - people >> aren't rejecting your position outright, but trying to get enough >> information to figure out how it's going to affect the entire >> language. > > Yes, I accept the responsibility for conflating what I see as a problem > with a possible solution to that problem (the two were inextricably > linked in my mind). > >> That's another place where the Python community differs from other >> language communities. We don't believe in features for the sake of >> features (whether you get them by adding or removing things); we >> believe the features in the language should interact well with each >> other. So while some feature may scratch a particular itch - say to do >> FP programming in Python - the fact that it turns ugly when used in >> non-FP code, or requires ignoring fundamental language layout rules, >> are not things that tie one hand behind your back, but crucial >> problems that must be solved if the feature is to be added. So even >> though the idea behind a feature may be generally acceptable - and >> that seems to be the case here - it won't be adopted until those >> details get worked out. > > Agreed, and I had intended to move to that stage once I felt there was a > consensus that there is actually something that needs solving. What I > failed to make clear in my position is that I'm not trying to add a > feature for a it's own sake, rather that the current process for > limiting the growth of the Python language is doomed to failure. The growth limiting mechanism is doomed to failure, or the language is doomed to failure? > I'm going to try to rephrase the fundamental problem I see here and > hopefully I won't fail so dismally this time: > > 1) I assert that small is better than large when it comes to limiting > complexity. It would seem that you are saying that limiting Python's complexity is a bad thing. Please correct me if my understanding of your statement is incorrect. I'm personally of the opinion that limiting the complexity of Python as a language is a good thing. And so far, I think Guido has done a fairly very good job at keeping Python moving in a positive direction. > 2) The primary method of limiting Python's core feature growth is for > the BDFL to dig in his heels and reject ideas until they are adequately > shown to be broadly needed and implementable. > > 3) This method has failed in the past*. In fact, I assert that this > method is guaranteed to fail unless all ideas that add syntactical > structures are rejected outright, regardless of their utility. It > cannot limit the growth of Python's core, it can only limit the rate at > which it grows. If the rate of language feature growth is slowed by Guido, then obviously the growth of Python's core language is limited. I would argue that this is a good thing, as the more language syntax and features that are present, the steeper the learning curve for new users of the language. Python 2.5.2 is a wholly different beast from the Python 1.5.2 that I started with, but I've had nearly 10 years to update my understanding of the language. For new users? Metaclasses, list comprehensions, generator expressions, generators, ... I get asked about them all. And the language keeps growing! > 4) As Python moves into new domains, as new programming languages come > into vogue, and as the art of programming itself advances (albeit > glacially), users will perceive needs in Python and will clamor for > extensions. Some of these will eventually be accepted. But not all. Why? Because not all proposed extensions to Python are worth discussing outside of a water-cooler "wouldn't it be neat if?" discussion. See the proposed extension to list comprehensions where "order by" was proposed from sql (a year or two ago at this point). That shouldn't have even made it to python-dev. Yet it did. Why? Because everyone has their "I really want X", and too few people are willing to acknowledge that they can be wrong. > 5) Given the added desire to maintain backwards-compatibility, old > features will not be shed as fast as new ones are added (unless that > becomes part of the process, which doesn't seem practicable to me). Python 3.0 is one of the ways that features are shed. While Guido has stated that he doesn't believe that there will be a Python 4k (aka Python 4.0) that hits the reset button again, I think that there needs to be one, so that Python *can* get rid of bad features that we were wrong about. There's also the PEP process, which as long as there isn't sweeping "break everything" changes, it would seem that we can remove features as time goes on. > 6) I believe that a large class of these features could be rendered moot > or redundant if the language embraced a more sweeping and fundamental > change. While this won't absolutely prevent the language's growth, it > provides a built-in deterrent. ...To a functional programming paradigm. That's what you meant to say, right? > There's an old quote by Larry Wall: "Perl is ugly because people wanted > it that way". Perl took the same approach to limiting features that > Python does, with the notable difference that Larry didn't reject ideas > as consistently as Guido (or apparently at all). Nevertheless, we're > now simply discussing the *rate* at which the language becomes large and > inconsistent rather than whether it will or not. > > If we agree that the issues I outline above are valid, then I think we > can start bickering over possible solutions and how those solutions > would affect Python on the whole. I don't agree. From rhamph at gmail.com Fri Sep 12 20:38:13 2008 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 12 Sep 2008 12:38:13 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221234951.8492.40.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> Message-ID: On Fri, Sep 12, 2008 at 9:55 AM, Cliff Wells wrote: > 1) I assert that small is better than large when it comes to limiting > complexity. We seem to have a fundamental disagreement here. I insist your changes would add substantial complexity with little benefit, primarily adding more ways to accomplish things that can easily be already accomplished. We are at an impasse. -- Adam Olsen, aka Rhamphoryncus From cliff at develix.com Fri Sep 12 21:29:49 2008 From: cliff at develix.com (Cliff Wells) Date: Fri, 12 Sep 2008 12:29:49 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> Message-ID: <1221247789.8492.109.camel@portable-evil> On Fri, 2008-09-12 at 09:51 -0700, Josiah Carlson wrote: > On Fri, Sep 12, 2008 at 8:55 AM, Cliff Wells wrote: > > Agreed, and I had intended to move to that stage once I felt there was a > > consensus that there is actually something that needs solving. What I > > failed to make clear in my position is that I'm not trying to add a > > feature for a it's own sake, rather that the current process for > > limiting the growth of the Python language is doomed to failure. > > The growth limiting mechanism is doomed to failure, or the language is > doomed to failure? I'm asserting that the limiting mechanism is doomed to failure. > > I'm going to try to rephrase the fundamental problem I see here and > > hopefully I won't fail so dismally this time: > > > > 1) I assert that small is better than large when it comes to limiting > > complexity. > > It would seem that you are saying that limiting Python's complexity is > a bad thing. Please correct me if my understanding of your statement > is incorrect. No, I'm saying limiting complexity is good and that smaller is better. > I'm personally of the opinion that limiting the complexity of Python > as a language is a good thing. And so far, I think Guido has done a > fairly very good job at keeping Python moving in a positive direction. Yes, but I also feel that the language has already grown to a point where only a shrinking subset of Python users fully know and understand the full feature set. For people who have been using Python for many years, they've possibly had time to digest the new features. I don't believe new users fare as well. > > 2) The primary method of limiting Python's core feature growth is for > > the BDFL to dig in his heels and reject ideas until they are adequately > > shown to be broadly needed and implementable. > > > > 3) This method has failed in the past*. In fact, I assert that this > > method is guaranteed to fail unless all ideas that add syntactical > > structures are rejected outright, regardless of their utility. It > > cannot limit the growth of Python's core, it can only limit the rate at > > which it grows. > > If the rate of language feature growth is slowed by Guido, then > obviously the growth of Python's core language is limited. A slow boat still reaches its destination. My position is that by obviating the need for a large class of syntax extensions, the onus is moved from the Python core team onto the user desiring a particular extension and by doing so, things like community pressure to add X can be reduced and by extension, the likelihood of the process folding under that pressure becomes much less likely. > I would argue that this is a good thing, as the more language syntax and > features that are present, the steeper the learning curve for new > users of the language. Absolutely. But this isn't what Python has been doing (as you acknowledge below). We seem to be agreeing on this point, so I believe our disagreement must center around whether the current process continues to be sufficient. > Python 2.5.2 is a wholly different beast from the Python 1.5.2 that I > started with, but I've had nearly 10 years to update my understanding > of the language. For new users? Metaclasses, list comprehensions, > generator expressions, generators, ... I get asked about them all. > And the language keeps growing! I switched from C to Python right as 1.5.1 was being phased out. To this day I continue to use a subset of Python that is probably much closer to 1.5.x than 2.5. This may be attributed to a personal failing (my self-professed laziness), but the fact remains that I prefer a small, versatile set of constructs to a large set of specialized ones. C, for all of its shortcomings, had one absolutely compelling feature: you could keep the entire language in your head and you could do it in an amazingly short time. Python 1.5 was similar in scope. 2.5 is arguably at or beyond that limit for most people. > > 4) As Python moves into new domains, as new programming languages come > > into vogue, and as the art of programming itself advances (albeit > > glacially), users will perceive needs in Python and will clamor for > > extensions. Some of these will eventually be accepted. > > But not all. Why? Because not all proposed extensions to Python are > worth discussing outside of a water-cooler "wouldn't it be neat if?" > discussion. See the proposed extension to list comprehensions where > "order by" was proposed from sql (a year or two ago at this point). > That shouldn't have even made it to python-dev. Yet it did. Why? > Because everyone has their "I really want X", and too few people are > willing to acknowledge that they can be wrong. Right, but I am suggesting that perhaps at least some of this problem (the need users feel for extensions) is currently built into the language. I also made this point to bolster my argument that feature-creep is inevitable. Outside of the object unification that's been happening over the last few years, most of the features being added to Python appear to be oriented to supporting the FP paradigm (generators, list comps, ternary if, etc). However these features appear to be added in an ad hoc fashion. I'm not suggesting they weren't thought out, only that they were added based on demand rather than as part of an overall plan as the object unification was. It appears to me that most of these "water-cooler" suggestions are proposed precisely because a) Users encounter a situation where an elegant solution isn't obvious and they feel they must resort to magic and hackery. b) The elegant solution they were looking for is embodied as an extension or requires knowledge that is more advanced than the problem at hand would seem to require (e.g. metaclasses). c) The PEPs (or more commonly, the flamewars on c.l.py) begin. For example, earlier in this thread you provided an elegant solution to what appeared (to myself and at least one other person) to be an intractable problem. But of course, it wasn't the natural solution to most people. It's ironic, because I've actually used a similar solution before, but wasn't able to see it this time. In fact, every time I have to use it, I have to reinvent it because it isn't the natural way the problem is expressed. The problem is most naturally expressed procedurally but must be solved with an OO paradigm. > > 5) Given the added desire to maintain backwards-compatibility, old > > features will not be shed as fast as new ones are added (unless that > > becomes part of the process, which doesn't seem practicable to me). > > Python 3.0 is one of the ways that features are shed. While Guido has > stated that he doesn't believe that there will be a Python 4k (aka > Python 4.0) that hits the reset button again, I think that there needs > to be one, so that Python *can* get rid of bad features that we were > wrong about. There's also the PEP process, which as long as there > isn't sweeping "break everything" changes, it would seem that we can > remove features as time goes on. Sure, and I think this is a valuable process. > > 6) I believe that a large class of these features could be rendered moot > > or redundant if the language embraced a more sweeping and fundamental > > change. While this won't absolutely prevent the language's growth, it > > provides a built-in deterrent. > > ...To a functional programming paradigm. That's what you meant to say, right? Yes, although by "to" I would mean "allow for" rather than "convert to". In other words, not convert to or even encourage the FP paradigm, but also not prevent it. It would be embracing multi-paradigm approaches rather than abandoning one in favor of another. Python has always been multi-paradigm to an extent (procedural and object), but has always been somewhat adverse to the functional paradigm (aside from a few helpful things). Before Python's object unification efforts, many OO purists declaimed Python because its hybrid approach. Pythonistas claimed that they were wrong because "practicality beats purity", etc. But of course, we've all seen that to a large degree they were right. Practicality might beat purity, but practicality plus purity trumps both. Functional programming is, in fact, simply the logical extension of procedural programming (or more precisely, procedural is a subset of functional). They are in no way mutually exclusive. Much as Python's OO was hampered by distinctions between builtin classes and user-defined classes (amongst other things), so is Python's procedural support hampered by it's rejection of the functional paradigm. > > There's an old quote by Larry Wall: "Perl is ugly because people wanted > > it that way". Perl took the same approach to limiting features that > > Python does, with the notable difference that Larry didn't reject ideas > > as consistently as Guido (or apparently at all). Nevertheless, we're > > now simply discussing the *rate* at which the language becomes large and > > inconsistent rather than whether it will or not. > > > > If we agree that the issues I outline above are valid, then I think we > > can start bickering over possible solutions and how those solutions > > would affect Python on the whole. > > I don't agree. Well, that's a start. Regards, Cliff From rhamph at gmail.com Fri Sep 12 22:24:05 2008 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 12 Sep 2008 14:24:05 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221247789.8492.109.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> <1221247789.8492.109.camel@portable-evil> Message-ID: On Fri, Sep 12, 2008 at 1:29 PM, Cliff Wells wrote: > A slow boat still reaches its destination. My position is that by > obviating the need for a large class of syntax extensions, the onus is > moved from the Python core team onto the user desiring a particular > extension and by doing so, things like community pressure to add X can > be reduced and by extension, the likelihood of the process folding under > that pressure becomes much less likely. Again, a vast difference in what we consider complexity. For a user, does it matter that we use "len(foo)" rather than "len foo"? A little bit, but it's not huge. For the language developers, does it matter? Again, it's a small benefit. It's still our job to design it, not a third-party. Syntax and stdlib are both part of the language. All the things you think we might avoid by being more "functional", we'd still have to do them, saving nothing. Except all the details suggest *worse* solutions would be found, as we wouldn't be fine-tuning the syntax for the use-cases. The only genuine use-case for your changes I've seen in this whole thread is the dispatch-dict, but your changes are far too distract to justify it. As an aside, I can't speak for Guido, but I haven't seen any indication python's development process may collapse. It's a non-problem. -- Adam Olsen, aka Rhamphoryncus From cliff at develix.com Fri Sep 12 22:34:00 2008 From: cliff at develix.com (Cliff Wells) Date: Fri, 12 Sep 2008 13:34:00 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <20080912140354.0cd694f6@bhuda.mired.org> References: <1221072180.12242.20.camel@portable-evil> <1221086389.12242.116.camel@portable-evil> <20080911104433.0406fcdc@bhuda.mired.org> <1221154829.12242.413.camel@portable-evil> <20080911134818.45d0ea38@bhuda.mired.org> <1221157164.12242.437.camel@portable-evil> <20080911145615.28de775f@bhuda.mired.org> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> <20080912140354.0cd694f6@bhuda.mired.org> Message-ID: <1221251640.8492.149.camel@portable-evil> On Fri, 2008-09-12 at 14:03 -0400, Mike Meyer wrote: > On Fri, 12 Sep 2008 08:55:51 -0700 > Cliff Wells wrote: > > 2) The primary method of limiting Python's core feature growth is for > > the BDFL to dig in his heels and reject ideas until they are adequately > > shown to be broadly needed and implementable. > > I don't think you have accurately described things, but it's > irrelevant because changing this doesn't change the nature of the > problem. It may be grossly oversimplified, but I think ultimately the process ends this way. > > 3) This method has failed in the past*. In fact, I assert that this > > method is guaranteed to fail unless all ideas that add syntactical > > structures are rejected outright, regardless of their utility. It > > cannot limit the growth of Python's core, it can only limit the rate at > > which it grows. > > I think you're using "fail" in two different contexts here; one for an > individual case - in that something was added - and one in the global > case - in that the language will continue to grow without bounds. I'm claiming that the global case is an inevitable conclusion of repeatedly catering to the individual case. > We'll leave aside the first version - whether some particular feature > was a failure or not is really a question of personal taste, and > changeable - and look at the latter. > > You're right, they can only limit the growth rate. I contend that that > isn't a failure - at least not until the language starts losing users > because the feature count has gotten too large. Based on the success > of languages where features are treated as valuable just by being > features, we're still a very long way from that point. I won't disagree that Python is still a long way from that point. I also want absolutely to distance myself from any claim that Python will one day become obsolete or even lose a single user because of this. First of all because it's flamebait and secondly because I don't believe it to be true. What I will claim is that the growing the language in such a way is not doing a favor to its users (even if they might think so for their particular extension). I don't think any of us disagree with that. Of course it's mildly ironic that I'm being told the same thing about my proposal, but what I am trying to convey is that it's my belief that my one modification helps eliminate the need for dozens of others. > > 5) Given the added desire to maintain backwards-compatibility, old > > features will not be shed as fast as new ones are added (unless that > > becomes part of the process, which doesn't seem practicable to me). > > This has already become part of the process. Python 3.0 can break > backwards compatibility, and so is shedding features - or at least > moving them out of the core. For instance, the builtin functions that > are now subsumed into list comprehensions are gone. It was originally > scheduled for release late this month, but it looks like the schedule > has slipped a couple of weeks. Yes, but we want to keep watershed instances like these to an absolute minimum. I think removing the redundant (and less flexible) variations that listcomp now subsumes is a good thing. What I'm saying is "let's do it again". Except what would be subsumed isn't just a handful of functions, but rather a whole class of existing and future language extensions. > > 6) I believe that a large class of these features could be rendered moot > > or redundant if the language embraced a more sweeping and fundamental > > change. While this won't absolutely prevent the language's growth, it > > provides a built-in deterrent. > > This may well be true. But the absolutely critical issue is that it > not break the fundamental nature of language. I.e. - if you have to > add delimters to replace indentation, you ain't going anywhere. Absolutely, which is why I've never proposed it. It would fundamentally alter the visual appearance of the language and ultimately isn't necessary anyway. > > There's an old quote by Larry Wall: "Perl is ugly because people wanted > > it that way". Perl took the same approach to limiting features that > > Python does, with the notable difference that Larry didn't reject ideas > > as consistently as Guido (or apparently at all). Nevertheless, we're > > now simply discussing the *rate* at which the language becomes large and > > inconsistent rather than whether it will or not. > > LW used to sign books with the note "There's more than one way to do > it". As far as I can tell, the Perl community never met a feature it > didn't like, and they're all in Perl5 - and Perl6 is going to make CL > look small. > > But you just slipped in a new word that you haven't used before - > "inconsistent". Becoming large does not necessarily make things > inconsistent. Most of the features added to Python don't make it > inconsistent. Actually, the nastiest problems with some of them are > *because* they are consistent with the rest of the language(*). True, I was thinking more of Perl when that word popped out of my keyboard. However, I think that there is a fundamental *logical* inconsistency in Python. The language consistently adheres to this inconsistency =) but it's there nonetheless. This was my motivation for attempting to demonstrate that the two forms of "if" currently in Python (statement and operator) could be logically combined into one if the restrictions on the former were lifted (see below). > Maintaining consistency is far more important than limiting size, and > is why most proposals are rejected. I don't feel that any of the > recent changes have introduced inconsistencies in the language. Ah, but I do: if cond: block vs expr if cond Here we have two forms of the *very same* logical construction. A programmer must select one or the other based on *context*. This is my definition of inconsistent (although it's roots lie in a deeper inconsistency, namely the distinction between expressions and statements). Regards, Cliff From rhamph at gmail.com Fri Sep 12 22:52:52 2008 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 12 Sep 2008 14:52:52 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221251640.8492.149.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> <20080912140354.0cd694f6@bhuda.mired.org> <1221251640.8492.149.camel@portable-evil> Message-ID: On Fri, Sep 12, 2008 at 2:34 PM, Cliff Wells wrote: > On Fri, 2008-09-12 at 14:03 -0400, Mike Meyer wrote: >> Maintaining consistency is far more important than limiting size, and >> is why most proposals are rejected. I don't feel that any of the >> recent changes have introduced inconsistencies in the language. > > Ah, but I do: > > if cond: block > > vs > > expr if cond > > Here we have two forms of the *very same* logical construction. A > programmer must select one or the other based on *context*. This is my > definition of inconsistent (although it's roots lie in a deeper > inconsistency, namely the distinction between expressions and > statements). By definition, anything more complex than a turing machine contains these redundancies (which you call inconsistencies). Clearly, we all use languages with a great number of them, so your notion that they're inherently bad is disproven. Which isn't to say they're inherently good either.. they do have a cost, increasing the mental load of the programmer. That suggests there's a tradeoff involved. Asking about the tradeoffs would be a much more productive discussion. -- Adam Olsen, aka Rhamphoryncus From grosser.meister.morti at gmx.net Sat Sep 13 00:57:30 2008 From: grosser.meister.morti at gmx.net (=?ISO-8859-15?Q?Mathias_Panzenb=F6ck?=) Date: Sat, 13 Sep 2008 00:57:30 +0200 Subject: [Python-ideas] xml generator that uses the with statement Message-ID: <48CAF3DA.5070402@gmx.net> Here is a quick and dirty draft of a xml generator that uses the with statement: http://twoday.tuwien.ac.at/pub/files/XmlMarkup (ZIP, 3 KB) It is inspired by rubies XmlMarkup class. Brief usage: >>> from __future__ import with_statement >>> from XmlMarkup import * >>> import sys >>> >>> with XmlMarkup(sys.stdout) as xm: >>> with xm.root: >>> xm.text('foo') >>> with xm.prefixMapping('x','http://example.com/x'): >>> with xm.tag.ns('http://example.com/x'): >>> xm.comment('comment') >>> with xm['text']: >>> xm.text('bar') >>> with xm.tag(foo='bar',egg='spam'): >>> pass foobar I'm not 100% sure about some parts of the 'syntax', though. E.g. maybe change this: >>> with xm.tag(x='y').ns('...'): >>> with xm.tag: >>> ... into this: >>> with xm.tag('...',x='y'): >>> with xm.tag(): >>> ... This Syntax is more concise than those unhandy and error prone beginElement/endElement calls (endElement needs the name as argument, too!). This way you never forget to close a tag again. :) It even provides a simple way to embed arbitrary data: >>> with xm.text as text: >>> # text is a pseudo file object >>> with XmlMarkup(text) as xm2: >>> # things you generate with xm2 will be escaped And I added a way to generate DTDs: >>> with xm.dtd('foo') as dtd: >>> dtd.element('a',oneof('x','y',PCDATA)) >>> dtd.element('x',('egg','bacon',many('spam'))) >>> dtd.attlist('a',att1 = (CDATA, DEFAULT, 'value'), att2 = (ID, REQUIRED)) >>> dtd.entity('ent','value') What do you think? :) -panzi From josiah.carlson at gmail.com Sat Sep 13 01:38:44 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Fri, 12 Sep 2008 16:38:44 -0700 Subject: [Python-ideas] xml generator that uses the with statement In-Reply-To: <48CAF3DA.5070402@gmx.net> References: <48CAF3DA.5070402@gmx.net> Message-ID: On Fri, Sep 12, 2008 at 3:57 PM, Mathias Panzenb?ck wrote: > Here is a quick and dirty draft of a xml generator that uses the with statement: > http://twoday.tuwien.ac.at/pub/files/XmlMarkup (ZIP, 3 KB) > > It is inspired by rubies XmlMarkup class. > > Brief usage: >>>> from __future__ import with_statement >>>> from XmlMarkup import * >>>> import sys >>>> >>>> with XmlMarkup(sys.stdout) as xm: >>>> with xm.root: >>>> xm.text('foo') >>>> with xm.prefixMapping('x','http://example.com/x'): >>>> with xm.tag.ns('http://example.com/x'): >>>> xm.comment('comment') >>>> with xm['text']: >>>> xm.text('bar') >>>> with xm.tag(foo='bar',egg='spam'): >>>> pass > > foo xmlns:x="http://example.com/x">bar foo="bar" egg="spam"> > > I'm not 100% sure about some parts of the 'syntax', though. > E.g. maybe change this: > >>>> with xm.tag(x='y').ns('...'): >>>> with xm.tag: >>>> ... > > into this: > >>>> with xm.tag('...',x='y'): >>>> with xm.tag(): >>>> ... > > > > This Syntax is more concise than those unhandy and error prone > beginElement/endElement calls (endElement needs the name as argument, too!). > This way you never forget to close a tag again. :) > > It even provides a simple way to embed arbitrary data: > >>>> with xm.text as text: >>>> # text is a pseudo file object >>>> with XmlMarkup(text) as xm2: >>>> # things you generate with xm2 will be escaped > > And I added a way to generate DTDs: > >>>> with xm.dtd('foo') as dtd: >>>> dtd.element('a',oneof('x','y',PCDATA)) >>>> dtd.element('x',('egg','bacon',many('spam'))) >>>> dtd.attlist('a',att1 = (CDATA, DEFAULT, 'value'), att2 = (ID, REQUIRED)) >>>> dtd.entity('ent','value') > > > What do you think? :) I've done a similar thing using with (and for in Python 2.4 or before) in constructing wxPython GUI widget layouts. - Josiah From grosser.meister.morti at gmx.net Sat Sep 13 01:54:35 2008 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Sat, 13 Sep 2008 01:54:35 +0200 Subject: [Python-ideas] xml generator that uses the with statement In-Reply-To: References: <48CAF3DA.5070402@gmx.net> Message-ID: <48CB013B.4070305@gmx.net> Josiah Carlson schrieb: > I've done a similar thing using with (and for in Python 2.4 or before) > in constructing wxPython GUI widget layouts. > Yeah, I also did something similar using for back in 2.4. But I considered that as a hack. with is the way to go. :) -panzi From cliff at develix.com Sat Sep 13 05:34:19 2008 From: cliff at develix.com (Cliff Wells) Date: Fri, 12 Sep 2008 20:34:19 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221162306.12242.495.camel@portable-evil> <20080911162143.2101a624@bhuda.mired.org> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> <1221247789.8492.109.camel@portable-evil> Message-ID: <1221276859.31915.6.camel@portable-evil> On Fri, 2008-09-12 at 14:24 -0600, Adam Olsen wrote: > On Fri, Sep 12, 2008 at 1:29 PM, Cliff Wells wrote: > > A slow boat still reaches its destination. My position is that by > > obviating the need for a large class of syntax extensions, the onus is > > moved from the Python core team onto the user desiring a particular > > extension and by doing so, things like community pressure to add X can > > be reduced and by extension, the likelihood of the process folding under > > that pressure becomes much less likely. > > Again, a vast difference in what we consider complexity. Well there's two forms of complexity to consider: language complexity and user-program complexity. A language that's too simple (e.g. assembler or C) can lead to an explosion of complexity when actually used to develop a non-trivial program. I'd actually agree that language features help reduce program complexity. For instance even the simple case of Python lacking the equivalent of repeat..until or do..while loop forces a tiny bit of complexity (in the form of a couple extra lines of code and usually an extra exit point from a loop) into a user's programs. The complexity I'm referring to is the fact that Python's collection of statements are rapidly forking into two forms: the original statement form, and newer expression forms. Worse, in some cases these expression forms exist in several sub-forms (e.g. list comps, expression comps, generators, etc). These forms all have a single common *logical* operation in mind (looping), but mutate into other forms because they are too specialized. It's also worth noting that most of them use an inverted syntax from the original statement in order to help distinguish them. I'm claiming that a single more flexible construct could potentially be used in place of all of them (for-expression with yield), and further, I'm claiming this is a case where simplifying the language would not complicate the user's program and in many cases might help to simplify it. > For a user, does it matter that we use "len(foo)" rather than "len > foo"? A little bit, but it's not huge. Or more consistently foo.len() which we *almost* have except it's spelled __len__ :P I agree this is a minor nit, but also rather pointless as far as I can tell. > For the language developers, does it matter? Again, it's a small > benefit. It's still our job to design it, not a third-party. Syntax > and stdlib are both part of the language. > > All the things you think we might avoid by being more "functional", > we'd still have to do them, saving nothing. Except all the details > suggest *worse* solutions would be found, as we wouldn't be > fine-tuning the syntax for the use-cases. > > The only genuine use-case for your changes I've seen in this whole > thread is the dispatch-dict, but your changes are far too distract to > justify it. I'm not surprised. In fact, after programming in Python myself for a decade or so I'm surprised I was able to find any suitable example ;-) The Sapir-Whorf hypotheses predicts this. A Lisp or Haskell programmer would probably be much better armed than I to provide concrete examples. Even if you don't subscribe to Sapir-Whorf, I think you could probably agree that at least the lack of idioms for FP in Python would contribute to our mutual inability to visualize particularly useful examples. > As an aside, I can't speak for Guido, but I haven't seen any > indication python's development process may collapse. It's a > non-problem. I didn't mean the development process may collapse, I mean the rejection process may collapse due to pressure (and I believe it has already on occasion). Regards, Cliff From rhamph at gmail.com Sat Sep 13 05:56:55 2008 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 12 Sep 2008 21:56:55 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221276859.31915.6.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> <1221247789.8492.109.camel@portable-evil> <1221276859.31915.6.camel@portable-evil> Message-ID: On Fri, Sep 12, 2008 at 9:34 PM, Cliff Wells wrote: > On Fri, 2008-09-12 at 14:24 -0600, Adam Olsen wrote: >> On Fri, Sep 12, 2008 at 1:29 PM, Cliff Wells wrote: >> > A slow boat still reaches its destination. My position is that by >> > obviating the need for a large class of syntax extensions, the onus is >> > moved from the Python core team onto the user desiring a particular >> > extension and by doing so, things like community pressure to add X can >> > be reduced and by extension, the likelihood of the process folding under >> > that pressure becomes much less likely. >> >> Again, a vast difference in what we consider complexity. > > Well there's two forms of complexity to consider: language complexity > and user-program complexity. A language that's too simple (e.g. > assembler or C) can lead to an explosion of complexity when actually > used to develop a non-trivial program. > > I'd actually agree that language features help reduce program > complexity. For instance even the simple case of Python lacking the > equivalent of repeat..until or do..while loop forces a tiny bit of > complexity (in the form of a couple extra lines of code and usually an > extra exit point from a loop) into a user's programs. > > The complexity I'm referring to is the fact that Python's collection of > statements are rapidly forking into two forms: the original statement > form, and newer expression forms. Worse, in some cases these > expression forms exist in several sub-forms (e.g. list comps, expression > comps, generators, etc). These forms all have a single common > *logical* operation in mind (looping), but mutate into other forms > because they are too specialized. It's also worth noting that most of > them use an inverted syntax from the original statement in order to help > distinguish them. > I'm claiming that a single more flexible construct could potentially be > used in place of all of them (for-expression with yield), and further, > I'm claiming this is a case where simplifying the language would not > complicate the user's program and in many cases might help to simplify > it. At most, it's a *small* benefit to the language. It comes a tradeoff, just like a+b rather than add(a, b), or len(x) rather than x.len(). A little bit more syntax to learn, which increases the learning curve (but for easy enough syntax the underlying operation is more to learn anyway), and in exchange it makes things easier for a skilled programmer. You're claiming we don't have to make that tradeoff.. but you've done *nothing* to substantiate that. Even logix only adds more complexity, rather than removing and unifying existing syntax. We don't care about purity. Statements vs expressions.. functional vs imperative.. they're tools, pure and simple. We switch back and forth as appropriate. Most of the time quietly, subtly, such that you usually don't even notice we've done it. We like it that way. You'll be a lot more productive if you learn to relax, concerning yourself with use-cases, not ideals. -- Adam Olsen, aka Rhamphoryncus From cliff at develix.com Sat Sep 13 18:02:52 2008 From: cliff at develix.com (Cliff Wells) Date: Sat, 13 Sep 2008 09:02:52 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221167218.12242.524.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> <1221247789.8492.109.camel@portable-evil> <1221276859.31915.6.camel@portable-evil> Message-ID: <1221321772.31915.41.camel@portable-evil> On Fri, 2008-09-12 at 21:56 -0600, Adam Olsen wrote: > At most, it's a *small* benefit to the language. It comes a tradeoff, > just like a+b rather than add(a, b), or len(x) rather than x.len(). Okay, you kept providing this example and I kept brushing it off because I didn't see where you were leading. Now I do and now I realize that I've utterly failed to get the problem across. Let's use your example from above: a+b vs add(a, b) a + b is the *exact* semantical equivalent of add(a, b). They do the same thing, they can be used in the same contexts. The + operator is pure semantic sugar (and well-used sugar too). Now let's imagine things aren't quite so equivalent. a+b has the same result as add(a, b), except you can only use the + operator in one particular context: assignment. It cannot be used say, with a comparison operator. So in my imaginary Python, this is legal x = 1 + 2 but this is not if x < y + 1: pass and must instead be written if x < add(y, 1): pass Now lets make it even more of an exact comparison. Let's say the add(x,y) function didn't even exist. You have to write it yourself. Worse, writing it requires fairly deep knowledge of Python, and worse yet, you sometimes must write a different add(x,y) function depending on what context you want to use it in. At this point we've now made your example much closer in spirit to the problem I'm attempting to present. > You're claiming we don't have to make that tradeoff.. but you've done > *nothing* to substantiate that. Even logix only adds more complexity, > rather than removing and unifying existing syntax. And you've provided nothing to substantiate this (aside from this assertion). Also, if we combine statements into expressions, how does this not unify existing syntax? Combining two into one is practically the definition of "unify". > We don't care about purity. Statements vs expressions.. functional vs > imperative.. they're tools, pure and simple. We switch back and forth > as appropriate. Most of the time quietly, subtly, such that you > usually don't even notice we've done it. We like it that way. Who is this "we"? People who use Python? Am I not a person who uses Python (and for the better part of 10 years I might add). I'm not some outsider saying "can you add X from my previously favorite language Y so I can program the way I used to". I'm someone who, until maybe a year ago, was just as narrow-minded about Python's shortcomings as you are now. I mean seriously, has it occurred to you that it's possible that you simply aren't getting my point? > You'll be a lot more productive if you learn to relax, concerning > yourself with use-cases, not ideals. Thanks for the advice. Should I also read a good book, maybe consider a massage? Regards, Cliff From cliff at develix.com Sat Sep 13 19:33:19 2008 From: cliff at develix.com (Cliff Wells) Date: Sat, 13 Sep 2008 10:33:19 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221192508.12242.616.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <48C9C2AB.5030103@gmx.net> <1221182371.12242.570.camel@portable-evil> <1221192508.12242.616.camel@portable-evil> Message-ID: <1221327199.11790.19.camel@portable-evil> On Thu, 2008-09-11 at 21:08 -0700, Cliff Wells wrote: > As I've pointed out, this has already been done (although the project > appears dead): > > http://www.livelogix.net/logix/index.html And after some research this morning, I found out why the project is dead: the author has long since moved to Ruby (where something like Logix is rendered moot). Figures. Cliff From arnodel at googlemail.com Sat Sep 13 20:01:23 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 13 Sep 2008 19:01:23 +0100 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221081763.12242.58.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> Message-ID: <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> On 10 Sep 2008, at 22:22, Cliff Wells wrote: > On Wed, 2008-09-10 at 21:46 +0200, Christian Heimes wrote: >> Cliff Wells wrote: >>> Any thoughts on this? I'm sure it's been brought up before, but I >>> haven't found any definitive discussions on why this rather >>> arbitrary >>> design decision continues to hold in the face of a general migration >>> away from imperative languages (especially when it seems it could be >>> changed without much backwards-compatibility issues). >> >> Two thoughts: >> >> Please elaborate how you like to change the syntax of Python. > > No changes. Simply lifting of a particular restriction. > >> I like to >> see some concrete examples how your syntax would look like. I also >> like >> to know how your are planing to implement features like lazy >> evaluation. >> The if else ternary operator statement is evaluated lazy. The same >> construct as expression wouldn't be lazy any more. > > Why not? > > a = ( > if a > 1 then: > long_calculation() > else: > other_long_calculation() > ) > > Clearly only one of these blocks would be evaluated at runtime. So what does: a = (if False: 1) evaluate to? -- Arnaud From grosser.meister.morti at gmx.net Sat Sep 13 20:39:14 2008 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Sat, 13 Sep 2008 20:39:14 +0200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> Message-ID: <48CC08D2.5080800@gmx.net> Arnaud Delobelle schrieb: > > So what does: > > a = (if False: 1) > > evaluate to? > OT: I attended a lecture called "abstract machines" and we had to write a little VM (parser, bytecode interpreter) on paper during the test. It was a expression based language and there was a "if ... then ... else ..." AND a "if ... then ..." construct. I asked the Prof. exactly the same question you did. He said: "Oh... well... strike that out." He really didn't notice that this is a problem when he wrote the test. *g* :P (Sorry, for the noise.) -panzi From bruce at leapyear.org Sat Sep 13 21:27:06 2008 From: bruce at leapyear.org (Bruce Leban) Date: Sat, 13 Sep 2008 12:27:06 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221321772.31915.41.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> <1221247789.8492.109.camel@portable-evil> <1221276859.31915.6.camel@portable-evil> <1221321772.31915.41.camel@portable-evil> Message-ID: On Sat, Sep 13, 2008 at 9:02 AM, Cliff Wells wrote: > Let's use your example from above: a+b vs add(a, b) > > a + b is the *exact* semantical equivalent of add(a, b). They do the > same thing, they can be used in the same contexts. The + operator is > pure semantic sugar (and well-used sugar too). > First, + is *syntactic* sugar not semantic sugar. Syntactic sugar is something that changes the appearance but not the semantics. C uses semicolons and Python doesn't. Is that syntactic sugar? Can I just ask C to remove the "restriction" that semicolons are required without defining precisely what that means? If I do, then I end up with something like Javascript's semicolon rule which is confusing and error-prone. No thanks. Now let's imagine things aren't quite so equivalent. a+b has the same > result as add(a, b), except you can only use the + operator in one > particular context: assignment. It cannot be used say, with a comparison > operator. > > So in my imaginary Python, this is legal > > x = 1 + 2 > > but this is not > > if x < y + 1: pass > > and must instead be written > > if x < add(y, 1): pass This is a strawman. You hypothesize that + would only be usable in the context of "assignment". But that's not the context. The context is expression. The semantics are the same. Here's a real example: consider a language where +, *, etc. can only be used in the context of augmented assignment. That is: a += b c *= d etc. while I have to use functional syntax everywhere else: x = add(a,b) y = mul(c,d) etc. Now I propose improving this language by eliminating the "restriction" that +, * etc. can only be used in assignments. You ask me to define the syntax and semantics of these operators and I brush that off as unimportant. All I'm advocating is "lifting a restriction" in the language. I'm unifying syntax not adding complexity, right? No need to explain pesky details. No. That's ridiculous. You might be surprised when you use my new features that x = 2 * 3 + 4 y = 2 - 3 + 4 sets x to 14 and y to -5 but that's not my fault. (And lest you say this is a strawman and no one would consider any value other than 10 and 3, go look at http://tinyurl.com/27ttcy.) This is what you're doing. If you can't precisely define how the language is different with your proposed change, then it's not worth discussing. You can do that with a formal definition as I suggested or by actually implementing your changes as Guido suggested. You can't do it by insisting that no one understands what you're talking about. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhamph at gmail.com Sat Sep 13 21:41:36 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 13 Sep 2008 13:41:36 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221321772.31915.41.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> <1221247789.8492.109.camel@portable-evil> <1221276859.31915.6.camel@portable-evil> <1221321772.31915.41.camel@portable-evil> Message-ID: On Sat, Sep 13, 2008 at 10:02 AM, Cliff Wells wrote: > On Fri, 2008-09-12 at 21:56 -0600, Adam Olsen wrote: > >> At most, it's a *small* benefit to the language. It comes a tradeoff, >> just like a+b rather than add(a, b), or len(x) rather than x.len(). > > Okay, you kept providing this example and I kept brushing it off because > I didn't see where you were leading. Now I do and now I realize that > I've utterly failed to get the problem across. > > Let's use your example from above: a+b vs add(a, b) > > a + b is the *exact* semantical equivalent of add(a, b). They do the > same thing, they can be used in the same contexts. The + operator is > pure semantic sugar (and well-used sugar too). > > Now let's imagine things aren't quite so equivalent. a+b has the same > result as add(a, b), except you can only use the + operator in one > particular context: assignment. It cannot be used say, with a comparison > operator. > > So in my imaginary Python, this is legal > > x = 1 + 2 > > but this is not > > if x < y + 1: pass > > and must instead be written > > if x < add(y, 1): pass > > Now lets make it even more of an exact comparison. > > Let's say the add(x,y) function didn't even exist. You have to write it > yourself. Worse, writing it requires fairly deep knowledge of Python, > and worse yet, you sometimes must write a different add(x,y) function > depending on what context you want to use it in. > > At this point we've now made your example much closer in spirit to the > problem I'm attempting to present. Your explanation would be true, except custom functions can be used as either statements or expressions! It is only the existing statements which are limited to being used as statements, and they're syntactic sugar. >> You're claiming we don't have to make that tradeoff.. but you've done >> *nothing* to substantiate that. Even logix only adds more complexity, >> rather than removing and unifying existing syntax. > > And you've provided nothing to substantiate this (aside from this > assertion). > > Also, if we combine statements into expressions, how does this not unify > existing syntax? Combining two into one is practically the definition > of "unify". Merging generator expressions, list comprehensions, and the for-loop would be unifying. Making the for-loop also usable as an expression is generalizing a specific feature. I've given many examples of why that doesn't work, so I'll assuming not seeing them was due to the terminology confusion. -- Adam Olsen, aka Rhamphoryncus From cliff at develix.com Sat Sep 13 23:58:00 2008 From: cliff at develix.com (Cliff Wells) Date: Sat, 13 Sep 2008 14:58:00 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221181315.12242.552.camel@portable-evil> <20080912110014.18a1029d@bhuda.mired.org> <1221234951.8492.40.camel@portable-evil> <1221247789.8492.109.camel@portable-evil> <1221276859.31915.6.camel@portable-evil> <1221321772.31915.41.camel@portable-evil> Message-ID: <1221343080.11790.62.camel@portable-evil> On Sat, 2008-09-13 at 12:27 -0700, Bruce Leban wrote: > > > On Sat, Sep 13, 2008 at 9:02 AM, Cliff Wells > wrote: > Let's use your example from above: a+b vs add(a, b) > > > a + b is the *exact* semantical equivalent of add(a, b). They > do the > same thing, they can be used in the same contexts. The + > operator is > pure semantic sugar (and well-used sugar too). > > > First, + is *syntactic* sugar not semantic sugar. Syntactic sugar is > something that changes the appearance but not the semantics. Sorry, clearly typing too fast. I meant semantic the first time I used it and syntactic the second. > C uses semicolons and Python doesn't. Python does, but only as a substitute for a newline. > Is that syntactic sugar? Can I just ask C to remove the "restriction" > that semicolons are required without defining precisely what that > means? Arguably you'd end up with what Python does now. I think that's well-enough defined (C's for-loop being an exception, obviously). > If I do, then I end up with something like Javascript's semicolon rule > which is confusing and error-prone. No thanks. I'm not aware of JavaScript's semicolon rule. When I write JS I use it as a line-terminator. > Now let's imagine things aren't quite so equivalent. a+b has > the same > result as add(a, b), except you can only use the + operator in > one > particular context: assignment. It cannot be used say, with a > comparison > operator. > > So in my imaginary Python, this is legal > > x = 1 + 2 > > but this is not > > if x < y + 1: pass > > and must instead be written > > if x < add(y, 1): pass > > > This is a strawman. You hypothesize that + would only be usable in the > context of "assignment". But that's not the context. The context is > expression. The semantics are the same. I was attempting to use Adam's example (which I would have labeled a strawman, except I don't believe it was intentional) in order to convey the difference between what he was saying and what I was saying. If it fails, well, it's not the example I would have chosen, but I felt it might help Adam and I find common ground for debating on, so . > Here's a real example: consider a language where +, *, etc. can only > be used in the context of augmented assignment. That is: > a += b > c *= d > etc. while I have to use functional syntax everywhere else: > x = add(a,b) > y = mul(c,d) > etc. Now I propose improving this language by eliminating the > "restriction" that +, * etc. can only be used in assignments. Actually, this is the strawman. If you read back (yes, I'm asking a lot) you'll note that I asserted that statements are (or could be) semantically equivalent to expressions except they have a limited context. Any statement can be made into an expression (but not the reverse). T > You ask me to define the syntax and semantics of these operators and I > brush that off as unimportant. All I'm advocating is "lifting a > restriction" in the language. I'm unifying syntax not adding > complexity, right? No need to explain pesky details. > > > No. That's ridiculous. You might be surprised when you use my new > features that > x = 2 * 3 + 4 > y = 2 - 3 + 4 > sets x to 14 and y to -5 but that's not my fault. (And lest you say > this is a strawman and no one would consider any value other than 10 > and 3, go look at http://tinyurl.com/27ttcy.) It's a matter of defining ambiguity. APL does it by being strictly left-to-right and Python (and most other languages) defines it by a (admittedly more complicated) algebraic order of operations. > This is what you're doing. If you can't precisely define how the > language is different with your proposed change, then it's not worth > discussing. You can do that with a formal definition as I suggested or > by actually implementing your changes as Guido suggested. I think providing examples is enough for a general discussion of the idea. Further, if anyone were really curious beyond just discussing it in the abstract, they could try Logix (although you'll unfortunately need to have Python 2.4 to do so). I understand that any such change would need to be adequately defined. But I don't consider the discussion to have progressed to that point. If people do not even understand that expression-oriented languages provide any advantage (in fact, many of them apparently consider it a disadvantage), then there's little point in discussing what particular syntax this apparently useless change would take on. > You can't do it by insisting that no one understands what you're > talking about. When someone suggests that the distinction between expressions and statements is syntactic sugar on the same level as the + operator, then I have no choice but to assume they really don't understand what I'm talking about. Whether that's a failing on my part or theirs is an open question, but there's no doubt about the lack of understanding. Anyway, to be frank, once I discovered that the developer of Logix had moved to Ruby, I finally admitted to myself that I was being completely unreasonable in expecting Python to be any more than what it is. I've already made up my mind to move to a language I find more suitable. Regards, Cliff From cliff at develix.com Sun Sep 14 00:17:42 2008 From: cliff at develix.com (Cliff Wells) Date: Sat, 13 Sep 2008 15:17:42 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> Message-ID: <1221344262.11790.66.camel@portable-evil> On Sat, 2008-09-13 at 19:01 +0100, Arnaud Delobelle wrote: > On 10 Sep 2008, at 22:22, Cliff Wells wrote: > > > On Wed, 2008-09-10 at 21:46 +0200, Christian Heimes wrote: > >> Cliff Wells wrote: > >>> Any thoughts on this? I'm sure it's been brought up before, but I > >>> haven't found any definitive discussions on why this rather > >>> arbitrary > >>> design decision continues to hold in the face of a general migration > >>> away from imperative languages (especially when it seems it could be > >>> changed without much backwards-compatibility issues). > >> > >> Two thoughts: > >> > >> Please elaborate how you like to change the syntax of Python. > > > > No changes. Simply lifting of a particular restriction. > > > >> I like to > >> see some concrete examples how your syntax would look like. I also > >> like > >> to know how your are planing to implement features like lazy > >> evaluation. > >> The if else ternary operator statement is evaluated lazy. The same > >> construct as expression wouldn't be lazy any more. > > > > Why not? > > > > a = ( > > if a > 1 then: > > long_calculation() > > else: > > other_long_calculation() > > ) > > > > Clearly only one of these blocks would be evaluated at runtime. > > So what does: > > a = (if False: 1) > > evaluate to? That's a good question. This is one of those areas where a definition would need to be created. My inclination is to say None (much like a function with no return statement). Cliff From cesare.dimauro at a-tono.com Sun Sep 14 01:24:48 2008 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Sun, 14 Sep 2008 01:24:48 +0200 (CEST) Subject: [Python-ideas] Statements vs Expressions... why? Message-ID: <63489.151.53.159.38.1221348288.squirrel@webmail2.pair.com> > On Sat, 2008-09-13 at 23:58, Cliff Wells wrote: > I understand that any such change would need to be adequately defined. > But I don't consider the discussion to have progressed to that point. > If people do not even understand that expression-oriented languages > provide any advantage (in fact, many of them apparently consider it a > disadvantage), then there's little point in discussing what particular > syntax this apparently useless change would take on. Are you sure that calculating and returning a value for each statement that is interpreted as an expression is really an advantage? I find little cases that shows such an advantage on real world code (one of each is the ternary operator, that was introduced in Python 2.5 to fill that "hole"). Just to be clear, I think that returning a value after executing a for, while, and even an if statement/expression EVERY TIME would be of no pratical use except for very rare cases. Consider that you have to spend resources (CPU and memory) calculating values, that most of the time will be trashed because they will not be used. I know that we already have functions that work the same way: they always return something, defaulting to None when no return statement is used. That's because Python has only a "function" type (contrary to Pascal, which distinguish between functions and procedures), but I think that Guido opted to reduce the complexity of the language giving just one subroutine invocation type to use. Do we really need to slow down the language (at the cost of less readability too, because I'm strongly convinced that using statements like expressions will reduce it) for such limited "added value"? My 2 cents. Cesare From rhamph at gmail.com Sun Sep 14 03:47:09 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 13 Sep 2008 19:47:09 -0600 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221343080.11790.62.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221234951.8492.40.camel@portable-evil> <1221247789.8492.109.camel@portable-evil> <1221276859.31915.6.camel@portable-evil> <1221321772.31915.41.camel@portable-evil> <1221343080.11790.62.camel@portable-evil> Message-ID: On Sat, Sep 13, 2008 at 3:58 PM, Cliff Wells wrote: > On Sat, 2008-09-13 at 12:27 -0700, Bruce Leban wrote: >> You can't do it by insisting that no one understands what you're >> talking about. > > When someone suggests that the distinction between expressions and > statements is syntactic sugar on the same level as the + operator, then > I have no choice but to assume they really don't understand what I'm > talking about. Whether that's a failing on my part or theirs is an open > question, but there's no doubt about the lack of understanding. That much we can agree on. Throughout this thread very little if any progress has been made. I'm willing to continue on IRC if you'd like, but I won't clutter the list any more. -- Adam Olsen, aka Rhamphoryncus From cliff at develix.com Sun Sep 14 05:13:30 2008 From: cliff at develix.com (Cliff Wells) Date: Sat, 13 Sep 2008 20:13:30 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <63489.151.53.159.38.1221348288.squirrel@webmail2.pair.com> References: <63489.151.53.159.38.1221348288.squirrel@webmail2.pair.com> Message-ID: <1221362010.8823.31.camel@portable-evil> On Sun, 2008-09-14 at 01:24 +0200, Cesare Di Mauro wrote: > > On Sat, 2008-09-13 at 23:58, Cliff Wells wrote: > > > I understand that any such change would need to be adequately defined. > > But I don't consider the discussion to have progressed to that point. > > If people do not even understand that expression-oriented languages > > provide any advantage (in fact, many of them apparently consider it a > > disadvantage), then there's little point in discussing what particular > > syntax this apparently useless change would take on. > > Are you sure that calculating and returning a value for each statement > that is interpreted as an expression is really an advantage? I find little > cases that shows such an advantage on real world code (one of each is the > ternary operator, that was introduced in Python 2.5 to fill that "hole"). The particular use-case I personally have in mind is for the development of DSL's, where the domain being modeled is most naturally expressed as nested expressions. There are many other uses and frankly many of the things we currently use objects for could arguably be more naturally expressed functionally. Honestly, I think it's very difficult for a person who's grown accustomed to a procedural style to grasp the potential here (and I certainly include myself in this group, although I feel I'm starting to see the light). > Just to be clear, I think that returning a value after executing a for, > while, and even an if statement/expression EVERY TIME would be of no > pratical use except for very rare cases. I remember this wisdom being encoded in the old adage "Lisp programmers know the value of everything and the cost of nothing" which was a jab at the relative slowness of Lisp compared to C. However the same argument could be made for almost any aspect of a dynamic interpreted language. I don't have any supporting data, but I strongly suspect the cost would be almost immeasurable compared with the cost we've already incurred to have the features we consider indispensable. Also, it's worth noting that in the majority (actually, I'd expect nearly all) of cases the value is already calculated, it simply isn't returned. I think that overhead would be minimal. > Consider that you have to spend resources (CPU and memory) calculating > values, that most of the time will be trashed because they will not be > used. > > I know that we already have functions that work the same way: they always > return something, defaulting to None when no return statement is used. > That's because Python has only a "function" type (contrary to Pascal, > which distinguish between functions and procedures), but I think that > Guido opted to reduce the complexity of the language giving just one > subroutine invocation type to use. I believe that was a good choice. Speaking of Pascal (which was the second language I learned), I think there's a good parallel to be drawn. When I was learning pointers in Pascal, I struggled badly. I assumed it was because pointer manipulation was difficult. Then I learned C, and discovered that in that language, pointers were so natural that I picked up in a day what I couldn't put together properly in weeks or months in Pascal. Pascal wanted to abstract away the machine and how data is stored, which is anathema to pointers. C, on the other hand presented data in a way that made pointers natural. On the other hand, when I started doing OO programming in C (yes, it can be done, to a limited degree), I was again in the same boat. I was able to make it work, since I was much more expert in C than I ever was in Pascal, but overall the experience made me realize that OO support in a language is a good thing ;-) You can fake OO with structs and pointers but it feels like a hack, even though it's kind of a satisfying one. Today, you can do a fair amount of FP in Python, but it feels unnatural. Even if you get your code to work as you want it probably won't appear elegant. You can fake some FP constructs with OO but again, it's really only a workaround, no matter how clever it might make you feel (in fact, I'd go so far as to suggest that if your code makes you feel clever, then you've just performed a workaround). > Do we really need to slow down the language (at the cost of less > readability too, because I'm strongly convinced that using statements like > expressions will reduce it) for such limited "added value"? I think reduced readability is perhaps the strongest argument against FP style. However, even OO presents this danger and yet we embrace it fully. I wish I could remember who this quote is attributed to (and the exact phrasing for that matter): "The danger of the fully realized object is that it appears ordinary" OO presents a very arguable danger of making code impenetrable and yet we rarely experience this problem in real life, and in fact find quite the opposite: many things are most naturally expressed in an object paradigm. Given that expression-oriented languages haven't collapsed in large-scale applications (in fact they have flourished) should be fair indication this perceived danger doesn't necessarily have to be reality in this case either. Overall, I think a language should trust the programmer to know what he wants to do in his own program and that he's the one who's best able to decide what paradigm best suits the domain at hand. Regards, Cliff From cliff at develix.com Sun Sep 14 05:16:13 2008 From: cliff at develix.com (Cliff Wells) Date: Sat, 13 Sep 2008 20:16:13 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221234951.8492.40.camel@portable-evil> <1221247789.8492.109.camel@portable-evil> <1221276859.31915.6.camel@portable-evil> <1221321772.31915.41.camel@portable-evil> <1221343080.11790.62.camel@portable-evil> Message-ID: <1221362173.8823.35.camel@portable-evil> On Sat, 2008-09-13 at 19:47 -0600, Adam Olsen wrote: > On Sat, Sep 13, 2008 at 3:58 PM, Cliff Wells wrote: > > On Sat, 2008-09-13 at 12:27 -0700, Bruce Leban wrote: > >> You can't do it by insisting that no one understands what you're > >> talking about. > > > > When someone suggests that the distinction between expressions and > > statements is syntactic sugar on the same level as the + operator, then > > I have no choice but to assume they really don't understand what I'm > > talking about. Whether that's a failing on my part or theirs is an open > > question, but there's no doubt about the lack of understanding. > > That much we can agree on. > > Throughout this thread very little if any progress has been made. I'm > willing to continue on IRC if you'd like, but I won't clutter the list > any more. Fair enough. I'm actually willing to let it go at this point. If someone has a point to make or a question, I won't ignore them, but I'll refrain from indulging myself any further. And seriously, sincere thanks for the debate. As I mentioned early on, I didn't really expect this to go anywhere productive (except to the extent it let me get it off my chest), so I appreciate everyone's patience. Regards, Cliff From arnodel at googlemail.com Sun Sep 14 08:23:07 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 14 Sep 2008 07:23:07 +0100 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221344262.11790.66.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> Message-ID: <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> On 13 Sep 2008, at 23:17, Cliff Wells wrote: > On Sat, 2008-09-13 at 19:01 +0100, Arnaud Delobelle wrote: >> >> So what does: >> >> a = (if False: 1) >> >> evaluate to? > > That's a good question. This is one of those areas where a definition > would need to be created. My inclination is to say None (much like a > function with no return statement). > Assuming the return value of "None", I go back to an example I gave earlier: factors = for x in range(2, n): if n % x == 0: x This doesn't work as intended (filtering out the non-factors). How to make it work? The only way I can think of is to make (if 0: 1) return a special "non-value" which loops will then filter out. But then we all know what happens to non-values. So how would you solve this problem? > Cliff -- Arnaud From cliff at develix.com Sun Sep 14 08:36:39 2008 From: cliff at develix.com (Cliff Wells) Date: Sat, 13 Sep 2008 23:36:39 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> Message-ID: <1221374199.8823.55.camel@portable-evil> On Sun, 2008-09-14 at 07:23 +0100, Arnaud Delobelle wrote: > On 13 Sep 2008, at 23:17, Cliff Wells wrote: > > > On Sat, 2008-09-13 at 19:01 +0100, Arnaud Delobelle wrote: > >> > >> So what does: > >> > >> a = (if False: 1) > >> > >> evaluate to? > > > > That's a good question. This is one of those areas where a definition > > would need to be created. My inclination is to say None (much like a > > function with no return statement). > > > > Assuming the return value of "None", I go back to an example I gave > earlier: > > factors = for x in range(2, n): > if n % x == 0: > x > > This doesn't work as intended (filtering out the non-factors). How to > make it work? The only way I can think of is to make (if 0: 1) return > a special "non-value" which loops will then filter out. But then we > all know what happens to non-values. > > So how would you solve this problem? By writing it properly ;-) factors = for x in range ( 2, n ): if n % x == 0: yield x As I mentioned previously, in order to merge the concept of generator with a for-expression would require bringing in the yield keyword, just as it does now for generator functions. The example you gave would evaluate to None (or perhaps an empty list or generator - that's a detail that would take more consideration before defining it). Regards, Cliff From arnodel at googlemail.com Sun Sep 14 09:36:57 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 14 Sep 2008 08:36:57 +0100 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221374199.8823.55.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> Message-ID: <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> On 14 Sep 2008, at 07:36, Cliff Wells wrote: > On Sun, 2008-09-14 at 07:23 +0100, Arnaud Delobelle wrote: >> On 13 Sep 2008, at 23:17, Cliff Wells wrote: >> >>> On Sat, 2008-09-13 at 19:01 +0100, Arnaud Delobelle wrote: >>>> >>>> So what does: >>>> >>>> a = (if False: 1) >>>> >>>> evaluate to? >>> >>> That's a good question. This is one of those areas where a >>> definition >>> would need to be created. My inclination is to say None (much >>> like a >>> function with no return statement). >>> >> >> Assuming the return value of "None", I go back to an example I gave >> earlier: >> >> factors = for x in range(2, n): >> if n % x == 0: >> x >> >> This doesn't work as intended (filtering out the non-factors). How >> to >> make it work? The only way I can think of is to make (if 0: 1) >> return >> a special "non-value" which loops will then filter out. But then we >> all know what happens to non-values. >> >> So how would you solve this problem? > > By writing it properly ;-) > > factors = for x in range ( 2, n ): > if n % x == 0: > yield x > > As I mentioned previously, in order to merge the concept of generator > with a for-expression would require bringing in the yield keyword, > just > as it does now for generator functions. > > The example you gave would evaluate to None (or perhaps an empty > list or > generator - that's a detail that would take more consideration before > defining it). > OK, but this seems to me incompatible with current Python: def chain(I, J): for i in I: yield i for j in J: yield j Currently >>> '-'.join(chain('spam', 'eggs')) 's-p-a-m-e-g-g-s' With your proposal, the first *expression* (for i in I: yield i) will evaluate to something like iter(I) and then be discarded. Then the second *expression* (for j in J: yield j) will evaluate to something like iter(J) which will be discarded. So chain('spam', 'eggs') will return None. > > Regards, > Cliff > > -- Arnaud From cliff at develix.com Sun Sep 14 10:25:18 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 01:25:18 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> Message-ID: <1221380718.21780.13.camel@portable-evil> On Sun, 2008-09-14 at 08:36 +0100, Arnaud Delobelle wrote: > On 14 Sep 2008, at 07:36, Cliff Wells wrote: > > > On Sun, 2008-09-14 at 07:23 +0100, Arnaud Delobelle wrote: > >> On 13 Sep 2008, at 23:17, Cliff Wells wrote: > >> > >>> On Sat, 2008-09-13 at 19:01 +0100, Arnaud Delobelle wrote: > >>>> > >>>> So what does: > >>>> > >>>> a = (if False: 1) > >>>> > >>>> evaluate to? > >>> > >>> That's a good question. This is one of those areas where a > >>> definition > >>> would need to be created. My inclination is to say None (much > >>> like a > >>> function with no return statement). > >>> > >> > >> Assuming the return value of "None", I go back to an example I gave > >> earlier: > >> > >> factors = for x in range(2, n): > >> if n % x == 0: > >> x > >> > >> This doesn't work as intended (filtering out the non-factors). How > >> to > >> make it work? The only way I can think of is to make (if 0: 1) > >> return > >> a special "non-value" which loops will then filter out. But then we > >> all know what happens to non-values. > >> > >> So how would you solve this problem? > > > > By writing it properly ;-) > > > > factors = for x in range ( 2, n ): > > if n % x == 0: > > yield x > > > > As I mentioned previously, in order to merge the concept of generator > > with a for-expression would require bringing in the yield keyword, > > just > > as it does now for generator functions. > > > > The example you gave would evaluate to None (or perhaps an empty > > list or > > generator - that's a detail that would take more consideration before > > defining it). > > > > OK, but this seems to me incompatible with current Python: > > def chain(I, J): > for i in I: yield i > for j in J: yield j > > Currently > > >>> '-'.join(chain('spam', 'eggs')) > 's-p-a-m-e-g-g-s' > > With your proposal, the first *expression* (for i in I: yield i) will > evaluate to something like iter(I) and then be discarded. Then the > second *expression* (for j in J: yield j) will evaluate to something > like iter(J) which will be discarded. So chain('spam', 'eggs') will > return None. It seems you have me on this one. There's clearly other ways to do the same thing, but since backwards-compatibility is a prerequisite I'll have to concede the point. A potential solution would to be to use a different keyword than "yield" to separate current syntax from my proposed syntax (that is, distinguish expression-scoped yield from function-scoped yield), and just side-step the issue, but that seems unappealing to me. Regards, Cliff From cliff at develix.com Sun Sep 14 10:36:55 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 01:36:55 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221380718.21780.13.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> Message-ID: <1221381415.21780.17.camel@portable-evil> On Sun, 2008-09-14 at 01:25 -0700, Cliff Wells wrote: > On Sun, 2008-09-14 at 08:36 +0100, Arnaud Delobelle wrote: > > On 14 Sep 2008, at 07:36, Cliff Wells wrote: > > > > > On Sun, 2008-09-14 at 07:23 +0100, Arnaud Delobelle wrote: > > >> On 13 Sep 2008, at 23:17, Cliff Wells wrote: > > >> > > >>> On Sat, 2008-09-13 at 19:01 +0100, Arnaud Delobelle wrote: > > >>>> > > >>>> So what does: > > >>>> > > >>>> a = (if False: 1) > > >>>> > > >>>> evaluate to? > > >>> > > >>> That's a good question. This is one of those areas where a > > >>> definition > > >>> would need to be created. My inclination is to say None (much > > >>> like a > > >>> function with no return statement). > > >>> > > >> > > >> Assuming the return value of "None", I go back to an example I gave > > >> earlier: > > >> > > >> factors = for x in range(2, n): > > >> if n % x == 0: > > >> x > > >> > > >> This doesn't work as intended (filtering out the non-factors). How > > >> to > > >> make it work? The only way I can think of is to make (if 0: 1) > > >> return > > >> a special "non-value" which loops will then filter out. But then we > > >> all know what happens to non-values. > > >> > > >> So how would you solve this problem? > > > > > > By writing it properly ;-) > > > > > > factors = for x in range ( 2, n ): > > > if n % x == 0: > > > yield x > > > > > > As I mentioned previously, in order to merge the concept of generator > > > with a for-expression would require bringing in the yield keyword, > > > just > > > as it does now for generator functions. > > > > > > The example you gave would evaluate to None (or perhaps an empty > > > list or > > > generator - that's a detail that would take more consideration before > > > defining it). > > > > > > > OK, but this seems to me incompatible with current Python: > > > > def chain(I, J): > > for i in I: yield i > > for j in J: yield j > > > > Currently > > > > >>> '-'.join(chain('spam', 'eggs')) > > 's-p-a-m-e-g-g-s' > > > > With your proposal, the first *expression* (for i in I: yield i) will > > evaluate to something like iter(I) and then be discarded. Then the > > second *expression* (for j in J: yield j) will evaluate to something > > like iter(J) which will be discarded. So chain('spam', 'eggs') will > > return None. > > It seems you have me on this one. There's clearly other ways to do the > same thing, but since backwards-compatibility is a prerequisite I'll > have to concede the point. > > A potential solution would to be to use a different keyword than "yield" > to separate current syntax from my proposed syntax (that is, distinguish > expression-scoped yield from function-scoped yield), and just side-step > the issue, but that seems unappealing to me. Actually, a few more minutes of pondering got me a solution I don't find abhorrent. Let yield mean what it currently does. Instead, let "continue" accept an optional parameter (like "return"). Then your above example continues to work and my proposed change would look like: for i in j: continue i I haven't considered this too deeply, but it is at least consistent with "return" syntax and doesn't add a new keyword. Regards, Cliff From cliff at develix.com Sun Sep 14 10:44:24 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 01:44:24 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221381415.21780.17.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> Message-ID: <1221381864.21780.25.camel@portable-evil> On Sun, 2008-09-14 at 01:36 -0700, Cliff Wells wrote: > On Sun, 2008-09-14 at 01:25 -0700, Cliff Wells wrote: > > On Sun, 2008-09-14 at 08:36 +0100, Arnaud Delobelle wrote: > > > On 14 Sep 2008, at 07:36, Cliff Wells wrote: > > > > > > > On Sun, 2008-09-14 at 07:23 +0100, Arnaud Delobelle wrote: > > > >> On 13 Sep 2008, at 23:17, Cliff Wells wrote: > > > >> > > > >>> On Sat, 2008-09-13 at 19:01 +0100, Arnaud Delobelle wrote: > > > >>>> > > > >>>> So what does: > > > >>>> > > > >>>> a = (if False: 1) > > > >>>> > > > >>>> evaluate to? > > > >>> > > > >>> That's a good question. This is one of those areas where a > > > >>> definition > > > >>> would need to be created. My inclination is to say None (much > > > >>> like a > > > >>> function with no return statement). > > > >>> > > > >> > > > >> Assuming the return value of "None", I go back to an example I gave > > > >> earlier: > > > >> > > > >> factors = for x in range(2, n): > > > >> if n % x == 0: > > > >> x > > > >> > > > >> This doesn't work as intended (filtering out the non-factors). How > > > >> to > > > >> make it work? The only way I can think of is to make (if 0: 1) > > > >> return > > > >> a special "non-value" which loops will then filter out. But then we > > > >> all know what happens to non-values. > > > >> > > > >> So how would you solve this problem? > > > > > > > > By writing it properly ;-) > > > > > > > > factors = for x in range ( 2, n ): > > > > if n % x == 0: > > > > yield x > > > > > > > > As I mentioned previously, in order to merge the concept of generator > > > > with a for-expression would require bringing in the yield keyword, > > > > just > > > > as it does now for generator functions. > > > > > > > > The example you gave would evaluate to None (or perhaps an empty > > > > list or > > > > generator - that's a detail that would take more consideration before > > > > defining it). > > > > > > > > > > OK, but this seems to me incompatible with current Python: > > > > > > def chain(I, J): > > > for i in I: yield i > > > for j in J: yield j > > > > > > Currently > > > > > > >>> '-'.join(chain('spam', 'eggs')) > > > 's-p-a-m-e-g-g-s' > > > > > > With your proposal, the first *expression* (for i in I: yield i) will > > > evaluate to something like iter(I) and then be discarded. Then the > > > second *expression* (for j in J: yield j) will evaluate to something > > > like iter(J) which will be discarded. So chain('spam', 'eggs') will > > > return None. > > > > It seems you have me on this one. There's clearly other ways to do the > > same thing, but since backwards-compatibility is a prerequisite I'll > > have to concede the point. > > > > A potential solution would to be to use a different keyword than "yield" > > to separate current syntax from my proposed syntax (that is, distinguish > > expression-scoped yield from function-scoped yield), and just side-step > > the issue, but that seems unappealing to me. > > Actually, a few more minutes of pondering got me a solution I don't find > abhorrent. Let yield mean what it currently does. Instead, let > "continue" accept an optional parameter (like "return"). Then your > above example continues to work and my proposed change would look like: > > for i in j: > continue i > > I haven't considered this too deeply, but it is at least consistent with > "return" syntax and doesn't add a new keyword. I'm probably replying way too fast (in fact, I know I am), but I have two thoughts on this: 1) it seems to alter the semantics of "continue" too much when considered against current syntax, but... 2) with the new syntax, it seems not too bad because j = range(3) for i in j: i # evaluates to [] for i in j: continue # evaluates to [] for i in j: continue i # evaluates to [0,1,2] Overall I'm a bit torn on the idea. Thoughts? Regards, Cliff From cliff at develix.com Sun Sep 14 10:54:19 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 01:54:19 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221381864.21780.25.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> Message-ID: <1221382459.21780.30.camel@portable-evil> On Sun, 2008-09-14 at 01:44 -0700, Cliff Wells wrote: > > I'm probably replying way too fast (in fact, I know I am), but I have > two thoughts on this: > > 1) it seems to alter the semantics of "continue" too much when > considered against current syntax, but... > > 2) with the new syntax, it seems not too bad because > > j = range(3) > for i in j: i # evaluates to [] > for i in j: continue # evaluates to [] > for i in j: continue i # evaluates to [0,1,2] Bah, I knew I was replying too fast. I'm thinking that "continue" would be redefined to mean "yield value and continue" which means that for i in j: continue # evaluates to [ None, None, None ] not [] would seem the most consistent, but I fear it might be less practical (as it would create problems trying to use for/continue inside other expressions, although the effect when for/continue is used as a statement remains fine). Cliff > Overall I'm a bit torn on the idea. Thoughts? > > Regards, > Cliff > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From arnodel at googlemail.com Sun Sep 14 11:08:08 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 14 Sep 2008 10:08:08 +0100 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221381864.21780.25.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> Message-ID: <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> On 14 Sep 2008, at 09:44, Cliff Wells wrote: > On Sun, 2008-09-14 at 01:36 -0700, Cliff Wells wrote: >> On Sun, 2008-09-14 at 01:25 -0700, Cliff Wells wrote: >>> On Sun, 2008-09-14 at 08:36 +0100, Arnaud Delobelle wrote: >>>> On 14 Sep 2008, at 07:36, Cliff Wells wrote: >>>> >>>>> On Sun, 2008-09-14 at 07:23 +0100, Arnaud Delobelle wrote: >>>>>> On 13 Sep 2008, at 23:17, Cliff Wells wrote: >>>>>> >>>>>>> On Sat, 2008-09-13 at 19:01 +0100, Arnaud Delobelle wrote: >>>>>>>> >>>>>>>> So what does: >>>>>>>> >>>>>>>> a = (if False: 1) >>>>>>>> >>>>>>>> evaluate to? >>>>>>> >>>>>>> That's a good question. This is one of those areas where a >>>>>>> definition >>>>>>> would need to be created. My inclination is to say None (much >>>>>>> like a >>>>>>> function with no return statement). >>>>>>> >>>>>> >>>>>> Assuming the return value of "None", I go back to an example I >>>>>> gave >>>>>> earlier: >>>>>> >>>>>> factors = for x in range(2, n): >>>>>> if n % x == 0: >>>>>> x >>>>>> >>>>>> This doesn't work as intended (filtering out the non-factors). >>>>>> How >>>>>> to >>>>>> make it work? The only way I can think of is to make (if 0: 1) >>>>>> return >>>>>> a special "non-value" which loops will then filter out. But >>>>>> then we >>>>>> all know what happens to non-values. >>>>>> >>>>>> So how would you solve this problem? >>>>> >>>>> By writing it properly ;-) >>>>> >>>>> factors = for x in range ( 2, n ): >>>>> if n % x == 0: >>>>> yield x >>>>> >>>>> As I mentioned previously, in order to merge the concept of >>>>> generator >>>>> with a for-expression would require bringing in the yield keyword, >>>>> just >>>>> as it does now for generator functions. >>>>> >>>>> The example you gave would evaluate to None (or perhaps an empty >>>>> list or >>>>> generator - that's a detail that would take more consideration >>>>> before >>>>> defining it). >>>>> >>>> >>>> OK, but this seems to me incompatible with current Python: >>>> >>>> def chain(I, J): >>>> for i in I: yield i >>>> for j in J: yield j >>>> >>>> Currently >>>> >>>>>>> '-'.join(chain('spam', 'eggs')) >>>> 's-p-a-m-e-g-g-s' >>>> >>>> With your proposal, the first *expression* (for i in I: yield i) >>>> will >>>> evaluate to something like iter(I) and then be discarded. Then the >>>> second *expression* (for j in J: yield j) will evaluate to >>>> something >>>> like iter(J) which will be discarded. So chain('spam', 'eggs') >>>> will >>>> return None. >>> >>> It seems you have me on this one. There's clearly other ways to >>> do the >>> same thing, but since backwards-compatibility is a prerequisite I'll >>> have to concede the point. >>> >>> A potential solution would to be to use a different keyword than >>> "yield" >>> to separate current syntax from my proposed syntax (that is, >>> distinguish >>> expression-scoped yield from function-scoped yield), and just side- >>> step >>> the issue, but that seems unappealing to me. >> >> Actually, a few more minutes of pondering got me a solution I don't >> find >> abhorrent. Let yield mean what it currently does. Instead, let >> "continue" accept an optional parameter (like "return"). Then your >> above example continues to work and my proposed change would look >> like: >> >> for i in j: >> continue i >> >> I haven't considered this too deeply, but it is at least consistent >> with >> "return" syntax and doesn't add a new keyword. > > I'm probably replying way too fast (in fact, I know I am), but I have > two thoughts on this: > > 1) it seems to alter the semantics of "continue" too much when > considered against current syntax, but... > > 2) with the new syntax, it seems not too bad because > > j = range(3) > for i in j: i # evaluates to [] > for i in j: continue # evaluates to [] > for i in j: continue i # evaluates to [0,1,2] > > Overall I'm a bit torn on the idea. Thoughts? > Let's not call it continue, but YIELD for now: for i in J: YIELD i Now this won't work for nested loops. E.g. in current python def flatten(I): for J in I: for j in J: yield j >>> '-'.join(flatten(['spam', 'eggs'])) 's-p-a-m-e-g-g-s' Now say you want to write that inline with a for-expression: '-'.join( for J in I: for j in J: YIELD j ) That won't work because the j's will be accumulated in the inner loop and the outer loop won't accumulate anything, therefore returning an empty iterable. The flatten() example above works because the the scope of the yield statement is clearly defined by the enclosing def statement. To make it work, not only you need a special yield expression but you also need a special for expression: '-'.join( FOR J in I: for j in J: YIELD j ) Here it is clear what happens: the YIELD accumulates values in the FOR loop. Not very coder friendly though :) Now compare with the current syntax: '-'.join(j for J in I for j in J) > Regards, > Cliff > -- Arnaud From cliff at develix.com Sun Sep 14 11:10:42 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 02:10:42 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221382459.21780.30.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <1221382459.21780.30.camel@portable-evil> Message-ID: <1221383442.21780.35.camel@portable-evil> On Sun, 2008-09-14 at 01:54 -0700, Cliff Wells wrote: > On Sun, 2008-09-14 at 01:44 -0700, Cliff Wells wrote: > > > > > I'm probably replying way too fast (in fact, I know I am), but I have > > two thoughts on this: > > > > 1) it seems to alter the semantics of "continue" too much when > > considered against current syntax, but... > > > > 2) with the new syntax, it seems not too bad because > > > > j = range(3) > > for i in j: i # evaluates to [] > > for i in j: continue # evaluates to [] > > for i in j: continue i # evaluates to [0,1,2] > > Bah, I knew I was replying too fast. I'm thinking that "continue" would > be redefined to mean "yield value and continue" which means that > > for i in j: continue # evaluates to [ None, None, None ] not [] > > would seem the most consistent, but I fear it might be less practical > (as it would create problems trying to use for/continue inside other > expressions, although the effect when for/continue is used as a > statement remains fine). It would *have* to evaluate to an empty list otherwise this code: for i in range(10000000): continue would create a huge list as a side-effect. So the question is, does this seem too inconsistent? Clearly returning [None, None, None] fits nicely with how yield currently works but it's not going to work in this case. Cliff From cliff at develix.com Sun Sep 14 11:25:31 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 02:25:31 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> Message-ID: <1221384331.21780.48.camel@portable-evil> On Sun, 2008-09-14 at 10:08 +0100, Arnaud Delobelle wrote: > On 14 Sep 2008, at 09:44, Cliff Wells wrote: > > > > j = range(3) > > for i in j: i # evaluates to [] > > for i in j: continue # evaluates to [] > > for i in j: continue i # evaluates to [0,1,2] > > > Let's not call it continue, but YIELD for now: > > for i in J: YIELD i > > Now this won't work for nested loops. E.g. in current python > > def flatten(I): > for J in I: > for j in J: > yield j > > >>> '-'.join(flatten(['spam', 'eggs'])) > 's-p-a-m-e-g-g-s' > > Now say you want to write that inline with a for-expression: > > '-'.join( > for J in I: > for j in J: > YIELD j > ) > > That won't work because the j's will be accumulated in the inner loop > and the outer loop won't accumulate anything, therefore returning an > empty iterable. How about this way instead (since for-loop is now an expression): '-'.join( for j in ( for J in I: YIELD J ): YIELD j ) > Now compare with the current syntax: > > '-'.join(j for J in I for j in J) Certainly more clear and concise, but since (luckily for me this time) we're maintaining backwards-compatibility, that form would still be available. Cliff From stefan_ml at behnel.de Sun Sep 14 19:37:55 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 14 Sep 2008 19:37:55 +0200 Subject: [Python-ideas] xml generator that uses the with statement In-Reply-To: <48CB013B.4070305@gmx.net> References: <48CAF3DA.5070402@gmx.net> <48CB013B.4070305@gmx.net> Message-ID: Mathias Panzenb?ck wrote: > Josiah Carlson schrieb: >> I've done a similar thing using with (and for in Python 2.4 or before) >> in constructing wxPython GUI widget layouts. >> > > Yeah, I also did something similar using for back in 2.4. But I considered that > as a hack. with is the way to go. :) There was a similar proposal on comp.lang.python a couple of weeks ago. Check the archives. Stefan From arnodel at googlemail.com Sun Sep 14 20:28:41 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 14 Sep 2008 19:28:41 +0100 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221384331.21780.48.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> Message-ID: On 14 Sep 2008, at 10:25, Cliff Wells wrote: >> Now say you want to write that inline with a for-expression: >> >> '-'.join( >> for J in I: >> for j in J: >> YIELD j >> ) >> >> That won't work because the j's will be accumulated in the inner loop >> and the outer loop won't accumulate anything, therefore returning an >> empty iterable. > > How about this way instead (since for-loop is now an expression): > > '-'.join( > for j in ( for J in I: YIELD J ): YIELD j > ) After you've had a good night's sleep and when you look at this again, you'll definitely think that you were too hasty in replying :) -- Arnaud From josiah.carlson at gmail.com Sun Sep 14 21:36:17 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 14 Sep 2008 12:36:17 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> Message-ID: On Sun, Sep 14, 2008 at 11:28 AM, Arnaud Delobelle wrote: > > On 14 Sep 2008, at 10:25, Cliff Wells wrote: > >>> Now say you want to write that inline with a for-expression: >>> >>> '-'.join( >>> for J in I: >>> for j in J: >>> YIELD j >>> ) >>> >>> That won't work because the j's will be accumulated in the inner loop >>> and the outer loop won't accumulate anything, therefore returning an >>> empty iterable. >> >> How about this way instead (since for-loop is now an expression): >> >> '-'.join( >> for j in ( for J in I: YIELD J ): YIELD j >> ) > > After you've had a good night's sleep and when you look at this again, > you'll definitely think that you were too hasty in replying :) Agreed. For all of the semantic and syntactic gymnastics and discussion about how statements -> expressions would make Python a better language, all I can conclude from the above is "I'm glad Python doesn't do that." - Josiah From cliff at develix.com Sun Sep 14 21:51:19 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 12:51:19 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> Message-ID: <1221421879.21780.70.camel@portable-evil> On Sun, 2008-09-14 at 19:28 +0100, Arnaud Delobelle wrote: > On 14 Sep 2008, at 10:25, Cliff Wells wrote: > > >> Now say you want to write that inline with a for-expression: > >> > >> '-'.join( > >> for J in I: > >> for j in J: > >> YIELD j > >> ) > >> > >> That won't work because the j's will be accumulated in the inner loop > >> and the outer loop won't accumulate anything, therefore returning an > >> empty iterable. > > > > How about this way instead (since for-loop is now an expression): > > > > '-'.join( > > for j in ( for J in I: YIELD J ): YIELD j > > ) > > After you've had a good night's sleep and when you look at this again, > you'll definitely think that you were too hasty in replying :) > Ha! I knew I should have gone to bed earlier =) I = [ 'spam', 'eggs' ] '-'.join ( for J in I: YIELD ( for j in J: YIELD j ) ) Now if only I'd followed Guido's suggestion I'd actually be able to test before I post ;-) Cliff From cliff at develix.com Sun Sep 14 22:02:47 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 13:02:47 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> Message-ID: <1221422567.21780.79.camel@portable-evil> On Sun, 2008-09-14 at 12:36 -0700, Josiah Carlson wrote: > Agreed. For all of the semantic and syntactic gymnastics and > discussion about how statements -> expressions would make Python a > better language, all I can conclude from the above is "I'm glad Python > doesn't do that." Well, realize that being able to do something doesn't make it the right thing to do in a particular situation. I don't think Arnaud's goal here is to show that functional programming is bad, rather he's forcing me to work out whether or not it could be done without breaking existing syntax (and a fine job he's doing too). The examples we've been working through are testing specific cases. It doesn't mean it would be the recommended idiom for these cases. When I was testing the macro feature for Breve (a bad name for the feature, I now realize), I wrote intentionally horrific code, simply to test that the solution was general enough: http://breve.twisty-industries.com/snippets/macro-madness Would I ever do that in real life? Only for testing =) Cliff From cliff at develix.com Sun Sep 14 22:47:20 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 13:47:20 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221422567.21780.79.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> Message-ID: <1221425240.21780.102.camel@portable-evil> On Sun, 2008-09-14 at 13:02 -0700, Cliff Wells wrote: > On Sun, 2008-09-14 at 12:36 -0700, Josiah Carlson wrote: > > > Agreed. For all of the semantic and syntactic gymnastics and > > discussion about how statements -> expressions would make Python a > > better language, all I can conclude from the above is "I'm glad Python > > doesn't do that." Using Breve as an example again: by your reasoning, the mere existence of Breve should be reason enough not to support OO in Python (or at the very least, magic methods). Breve more or less a functional DSL that barely resembles Python but actually *is* Python (in fact, I've seen someone assert that Breve templates could not possibly be actual Python code). Even you were initially convinced that I was doing code-generation. Breve abuses classes and magic methods (albeit in an entirely legal way) to create the illusion of a declarative DSL. Clearly I think this ability is a good thing, but it could also be argued that what I've done is inscrutable and this type of code should not be allowed in Python. In fact, your own example of mimicking a dispatch "table" with a class is arguably object abuse. Personally I don't think it's a bad solution (especially in the absence of a better way), but you've basically mapped what is logically a functional problem onto an object. My point is that just because something *can* be abused, it isn't reason to throw the baby out with the proverbial bath water. The more general a tool is, the more able it is to be used incorrectly (as anyone who's pried a lid off a can of paint with a screwdriver can attest). Regards, Cliff From grosser.meister.morti at gmx.net Sun Sep 14 23:34:57 2008 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Sun, 14 Sep 2008 23:34:57 +0200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221072180.12242.20.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> Message-ID: <48CD8381.4040503@gmx.net> If I understand this right, then this would become legal too: x = if cond(a): return b So what value is assigned to x when cond(a) is True, what value when it is False? What value does return return? The thing is that return does definitively not return anything, but as a statement, it can be placed everywhere in a block. -panzi From cliff at develix.com Sun Sep 14 23:41:19 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 14:41:19 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CD8381.4040503@gmx.net> References: <1221072180.12242.20.camel@portable-evil> <48CD8381.4040503@gmx.net> Message-ID: <1221428479.21780.111.camel@portable-evil> On Sun, 2008-09-14 at 23:34 +0200, Mathias Panzenb?ck wrote: > If I understand this right, then this would become legal too: > > x = if cond(a): > return b return would always be a syntax error outside of a function. > > So what value is assigned to x when cond(a) is True, what value when it is > False? x = if False: 1 # evaluates to None x = if True: 1 # evalutates to True x = if True: return 1 # syntax error: return outside of function def foo(): x = if True: return 1 # exits the enclosing function. value of if-expression is moot as it's never reached. > What value does return return? The thing is that return does definitively > not return anything, but as a statement, it can be placed everywhere in a block. Not that it matters, but return always returns something, either a user-specified value or None. Regards, Cliff From cliff at develix.com Sun Sep 14 23:45:04 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 14:45:04 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221428479.21780.111.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <48CD8381.4040503@gmx.net> <1221428479.21780.111.camel@portable-evil> Message-ID: <1221428704.21780.114.camel@portable-evil> On Sun, 2008-09-14 at 14:41 -0700, Cliff Wells wrote: > On Sun, 2008-09-14 at 23:34 +0200, Mathias Panzenb?ck wrote: > > If I understand this right, then this would become legal too: > > > > x = if cond(a): > > return b > > return would always be a syntax error outside of a function. > > > > > So what value is assigned to x when cond(a) is True, what value when it is > > False? > > x = if False: 1 # evaluates to None > x = if True: 1 # evalutates to True I meant "evaluates to 1", of course. Cliff From grosser.meister.morti at gmx.net Sun Sep 14 23:51:02 2008 From: grosser.meister.morti at gmx.net (=?UTF-8?B?TWF0aGlhcyBQYW56ZW5iw7Zjaw==?=) Date: Sun, 14 Sep 2008 23:51:02 +0200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221428479.21780.111.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <48CD8381.4040503@gmx.net> <1221428479.21780.111.camel@portable-evil> Message-ID: <48CD8746.7040504@gmx.net> Cliff Wells schrieb: > Not that it matters, but return always returns something, either a > user-specified value or None. > Ok, I meant it never evaluates to some value. :P From grflanagan at gmail.com Mon Sep 15 00:49:46 2008 From: grflanagan at gmail.com (Gerard flanagan) Date: Mon, 15 Sep 2008 00:49:46 +0200 Subject: [Python-ideas] xml generator that uses the with statement In-Reply-To: <48CAF3DA.5070402@gmx.net> References: <48CAF3DA.5070402@gmx.net> Message-ID: Mathias Panzenb?ck wrote: > Here is a quick and dirty draft of a xml generator that uses the with statement: > http://twoday.tuwien.ac.at/pub/files/XmlMarkup (ZIP, 3 KB) > > It is inspired by rubies XmlMarkup class. > > Brief usage: >>>> from __future__ import with_statement >>>> from XmlMarkup import * >>>> import sys >>>> >>>> with XmlMarkup(sys.stdout) as xm: >>>> with xm.root: >>>> xm.text('foo') >>>> with xm.prefixMapping('x','http://example.com/x'): >>>> with xm.tag.ns('http://example.com/x'): >>>> xm.comment('comment') >>>> with xm['text']: >>>> xm.text('bar') >>>> with xm.tag(foo='bar',egg='spam'): >>>> pass > > foo xmlns:x="http://example.com/x">bar foo="bar" egg="spam"> > [...] > > What do you think? :) > > > -panzi Something similar, but without namespace handling: http://article.gmane.org/gmane.comp.python.general/579900/ (I like that you have a class called 'maybe' :-) Regards Ger From grosser.meister.morti at gmx.net Mon Sep 15 02:44:48 2008 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Mon, 15 Sep 2008 02:44:48 +0200 Subject: [Python-ideas] xml generator that uses the with statement In-Reply-To: References: <48CAF3DA.5070402@gmx.net> Message-ID: <48CDB000.3030206@gmx.net> > Something similar, but without namespace handling: > > http://article.gmane.org/gmane.comp.python.general/579900/ > > (I like that you have a class called 'maybe' :-) > > Regards > > Ger Hm, it seems that there are a lot of similar xml generators. I think one of them should be included in the python standard library. I'd say the one that 1. handles the xml standard best and 2. has the "best" (most concise, handy and complete) API. Hmm. There should be some kind of contest. :) -panzi From greg.ewing at canterbury.ac.nz Mon Sep 15 02:37:55 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Sep 2008 12:37:55 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221374199.8823.55.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> Message-ID: <48CDAE63.4000009@canterbury.ac.nz> Cliff Wells wrote: > factors = for x in range ( 2, n ): > if n % x == 0: > yield x Is this construct meant to be a list comprehension or a generator expression? If it's a GC, you just get factors bound to an iterator, not a list of results from that iterator. If it's an LC, then what do you do if you want a dict or tuple comprehension instead? Also, 'yield' already has a meaning that's different from what you seem to want here. Consider: def foo(n): for x in range ( 2, n ): if n % x == 0: yield x Here the whole function is a generator, and the yield causes an item to be returned to the generator's caller. Now, if there is no difference between a statement and an expression, the following should do exactly the same thing: def foo(n): factor = for x in range ( 2, n ): if n % x == 0: yield x -- Greg From greg.ewing at canterbury.ac.nz Mon Sep 15 02:50:41 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Sep 2008 12:50:41 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221380718.21780.13.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> Message-ID: <48CDB161.8060103@canterbury.ac.nz> Cliff Wells wrote: > A potential solution would to be to use a different keyword than "yield" > to separate current syntax from my proposed syntax But inserting any kind of keyword into LCs is going to make them ugly and verbose. The point of having LCs in the first place is that they express certain kinds of things very concisely. Adding keywords and colons to them messes that up. Likewise with conditional expressions. It would be disappointing if, instead of x = a if b else c we had to write x = if a: b else: c Colons in the middle of expressions look ugly, IMO. -- Greg From greg.ewing at canterbury.ac.nz Mon Sep 15 03:00:30 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Sep 2008 13:00:30 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221384331.21780.48.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> Message-ID: <48CDB3AE.6090704@canterbury.ac.nz> Cliff Wells wrote: > '-'.join( > for j in ( for J in I: YIELD J ): YIELD j > ) Noooooo..... this is getting worse and worse. You seem to be thinking of syntax issues as though they were purely technical puzzles. The're not -- they're at least as much human-factors issues. > Certainly more clear and concise, but since (luckily for me this time) > we're maintaining backwards-compatibility, that form would still be > available. But if you keep all the existing syntax as well, you haven't simplified anything. -- Greg From cliff at develix.com Mon Sep 15 03:16:18 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 18:16:18 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CDAE63.4000009@canterbury.ac.nz> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <48CDAE63.4000009@canterbury.ac.nz> Message-ID: <1221441378.21780.143.camel@portable-evil> On Mon, 2008-09-15 at 12:37 +1200, Greg Ewing wrote: > Cliff Wells wrote: > > > factors = for x in range ( 2, n ): > > if n % x == 0: > > yield x > > Is this construct meant to be a list comprehension or > a generator expression? If it's a GC, you just get > factors bound to an iterator, not a list of results > from that iterator. First thing, I've recanted reusing "yield" (see below), but your questions still hold in any case. It evaluates to an iterable, I think a generator being the logical choice (although I've been using "[]" to indicate an empty iterable in discussion for brevity). > If it's an LC, then what do you > do if you want a dict or tuple comprehension instead? Did we already grow those? I don't want to debate tuple comprehension (I admit I fail to see the point), but dict comprehensions would be great. To be clear, I'm aiming for as much backwards-compatibility as possible, so I'm in no way suggesting removing comprehensions (although I think they'd become somewhat redundant). In the hypothetical absence of a comprehension feature, I'd probably try to write one like these (I'm slipping "continue" in here as a replacement for yield, which I explain below): x = for i in j: if i: continue i else: continue i**2 # list comp x = dict ( for i in j: continue i, i**2 ) # not a dict comp ;-) > Also, 'yield' already has a meaning that's different > from what you seem to want here. Consider: Yes, that's why I had to retract using the yield keyword in this context. Instead I propose either a new keyword or allowing continue to accept an optional argument causing it to mean "yield expression and continue" rather than just "continue": factors = for x in range ( 2, n ): in n % x == 0: continue x continue without an argument would continue to mean what it does now. This raises some other concerns, but it's an option to be weighed against adding a keyword. In any case, using "yield" is straight out. Regards, Cliff From cliff at develix.com Mon Sep 15 03:32:12 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 18:32:12 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CDB3AE.6090704@canterbury.ac.nz> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <48CDB3AE.6090704@canterbury.ac.nz> Message-ID: <1221442332.21780.159.camel@portable-evil> On Mon, 2008-09-15 at 13:00 +1200, Greg Ewing wrote: > Cliff Wells wrote: > > > '-'.join( > > for j in ( for J in I: YIELD J ): YIELD j > > ) > Noooooo..... this is getting worse and worse. > > You seem to be thinking of syntax issues as though > they were purely technical puzzles. The're not -- > they're at least as much human-factors issues > Not that it's terribly relevant to what you say, but: '-'.join ( for J in I: YIELD ( for j in J: YIELD j ) ) is the corrected form (although YIELD continues to be a placeholder of course). In any case, my form does make it slightly more complicated for the simplest case, but makes it much less complicated for more complex cases (for the same reasons a plain for-statement can): x = [ J [ a ] for a in b if c ] vs x = for a in b: if c: continue J [ a ] The complexity has barely increased and yet the second form is already more readable (the formatting of the first reflects what I typically do as listcomps get more complex - overkill here but it demonstrates where I'm going). Given that the listcomp is the only direct form of looping available as an expression, I've written some really ugly looking ones. I've actually taken to commenting open/close brackets simply to help distinguish them. Not to mention, the listcomp's placement of the yielded result before the loop and condition only works well when it's a very simple expression. It doesn't scale well with complexity (not that I think it was meant to). > > Certainly more clear and concise, but since (luckily for me this time) > > we're maintaining backwards-compatibility, that form would still be > > available. > > But if you keep all the existing syntax as well, > you haven't simplified anything. Yes, that's unfortunate. It might, however, obviate the need for newer ones. Cliff From greg.ewing at canterbury.ac.nz Mon Sep 15 03:35:45 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Sep 2008 13:35:45 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221425240.21780.102.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> <1221425240.21780.102.camel@portable-evil> Message-ID: <48CDBBF1.3000303@canterbury.ac.nz> Cliff Wells wrote: > Breve more or less a functional DSL that barely resembles Python but > actually *is* Python (in fact, I've seen someone assert that Breve > templates could not possibly be actual Python code). If it looks that little like Python, I'd say it really is a different language, and you have no right to expect to be able to use the Python compiler as-is to process it. Rather than twisting Python to make it possible to abuse it even further, you'd be better off writing a Breve compiler in Python that produces Python code (or maybe even translates it directly to Python bytecode). -- Greg From santagada at gmail.com Mon Sep 15 03:58:37 2008 From: santagada at gmail.com (Leonardo Santagada) Date: Sun, 14 Sep 2008 22:58:37 -0300 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CDBBF1.3000303@canterbury.ac.nz> References: <1221072180.12242.20.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> <1221425240.21780.102.camel@portable-evil> <48CDBBF1.3000303@canterbury.ac.nz> Message-ID: <98483E4B-7A8A-4C22-9E3D-581C62D604E9@gmail.com> On Sep 14, 2008, at 10:35 PM, Greg Ewing wrote: > Cliff Wells wrote: > >> Breve more or less a functional DSL that barely resembles Python but >> actually *is* Python (in fact, I've seen someone assert that Breve >> templates could not possibly be actual Python code). > > If it looks that little like Python, I'd say it really > is a different language, and you have no right to expect > to be able to use the Python compiler as-is to process > it. > > Rather than twisting Python to make it possible to > abuse it even further, you'd be better off writing a > Breve compiler in Python that produces Python code > (or maybe even translates it directly to Python > bytecode). I was having fun reading the discussion, specially now that it got to the part were most people are starting to realise how ugly python would become, to put it lightly we got to the dificult part (the "how"). But all Cliff wanted is to have better support for DSL in python, that is his real use case. I say why don't we focus on it. The only clean way to support DSL in python in my view would be to support something like pyparsing or ANTLR and make it generate a parser and some simple form of compilation to python bytecodes. How can we support that? Is this even desired? -- Leonardo Santagada santagada at gmail.com From cliff at develix.com Mon Sep 15 04:04:54 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 19:04:54 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CDBBF1.3000303@canterbury.ac.nz> References: <1221072180.12242.20.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> <1221425240.21780.102.camel@portable-evil> <48CDBBF1.3000303@canterbury.ac.nz> Message-ID: <1221444294.21780.173.camel@portable-evil> On Mon, 2008-09-15 at 13:35 +1200, Greg Ewing wrote: > Cliff Wells wrote: > > > Breve more or less a functional DSL that barely resembles Python but > > actually *is* Python (in fact, I've seen someone assert that Breve > > templates could not possibly be actual Python code). > > If it looks that little like Python, I'd say it really > is a different language, and you have no right to expect > to be able to use the Python compiler as-is to process > it. Well *I* think it looks like Python, but then I understand (as can I think anyone familiar with Python's magic methods), but at least superficially it gives a different impression. In any case, Breve is mostly of interest in that it is the project that both made me appreciate the power of expressions and start bumping into Python's second-class support of them. > Rather than twisting Python to make it possible to > abuse it even further, you'd be better off writing a > Breve compiler in Python that produces Python code I think code generation is arguably worse than functional programming. > (or maybe even translates it directly to Python > bytecode). A seriously complicated and highly technical bit of code I'd likely have to fix every few years (and maintain multiple versions of to support multiple Python versions)? No thanks. In any case, I've already choked down my Python snobbery and ordered a Ruby book. Clearly where I want to go in programming isn't available in Python nor will it be any time soon. Regards, Cliff From cliff at develix.com Mon Sep 15 04:10:11 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 19:10:11 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <98483E4B-7A8A-4C22-9E3D-581C62D604E9@gmail.com> References: <1221072180.12242.20.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> <1221425240.21780.102.camel@portable-evil> <48CDBBF1.3000303@canterbury.ac.nz> <98483E4B-7A8A-4C22-9E3D-581C62D604E9@gmail.com> Message-ID: <1221444611.21780.178.camel@portable-evil> On Sun, 2008-09-14 at 22:58 -0300, Leonardo Santagada wrote: > I was having fun reading the discussion, specially now that it got to > the part were most people are starting to realise how ugly python > would become, to put it lightly we got to the dificult part (the "how"). > > But all Cliff wanted is to have better support for DSL in python, that > is his real use case. I say why don't we focus on it. > > The only clean way to support DSL in python in my view would be to > support something like pyparsing or ANTLR and make it generate a > parser and some simple form of compilation to python bytecodes. How > can we support that? Is this even desired? Actually you can do quite a lot with plain objects and class magic. But if your DSL isn't purely procedural (Breve is declarative-functional) then you are left without much in the way of flow control as most flow control in Python is done via statements. Overall I've already made a decision about my programming future, but certainly better DSL support in Python would be a good thing. Regards, Cliff From greg.ewing at canterbury.ac.nz Mon Sep 15 04:11:48 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Sep 2008 14:11:48 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221442332.21780.159.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <48CDB3AE.6090704@canterbury.ac.nz> <1221442332.21780.159.camel@portable-evil> Message-ID: <48CDC464.4020809@canterbury.ac.nz> Cliff Wells wrote: > Not that it's terribly relevant to what you say, but: > > '-'.join ( > for J in I: YIELD ( for j in J: YIELD j ) > ) No, I think you were right the first time. The above looks like it will generate a sequence of iterators, not a flat sequence of j values. > x = [ J [ a ] > for a in b > if c ] > > vs > > x = for a in b: > if c: continue J [ a ] > > The complexity has barely increased and yet the second form is already > more readable That's a matter of opinion. If you lay out the first one as x = [ J [ a ] for a in b if c ] there's not much difference between them. > Not to mention, the listcomp's placement of the > yielded result before the loop and condition only works well when it's a > very simple expression. It doesn't scale well with complexity (not that > I think it was meant to). You're right, it's not meant to. It's meant for the simple cases where all the syntactic overhead of a full for-statement and list appending code swamps the content. Given that, adding any extra syntactic baggage, even just a 'continue' keyword, reduces its effectiveness for its intended purpose. Also, I think it reads quite nicely with the expression at the beginning. It's modelled after the mathematical notation for describing sets: {x : some conditions on x} -- Greg From greg.ewing at canterbury.ac.nz Mon Sep 15 04:14:29 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Sep 2008 14:14:29 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221441378.21780.143.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <48CDAE63.4000009@canterbury.ac.nz> <1221441378.21780.143.camel@portable-evil> Message-ID: <48CDC505.9090007@canterbury.ac.nz> Cliff Wells wrote: > On Mon, 2008-09-15 at 12:37 +1200, Greg Ewing wrote: >>If it's an LC, then what do you >>do if you want a dict or tuple comprehension instead? > > Did we already grow those? I'm not actually sure. I think Py3 has dict and set comprehensions, but not tuple comprehensions. -- Greg From greg.ewing at canterbury.ac.nz Mon Sep 15 04:23:41 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Sep 2008 14:23:41 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <98483E4B-7A8A-4C22-9E3D-581C62D604E9@gmail.com> References: <1221072180.12242.20.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> <1221425240.21780.102.camel@portable-evil> <48CDBBF1.3000303@canterbury.ac.nz> <98483E4B-7A8A-4C22-9E3D-581C62D604E9@gmail.com> Message-ID: <48CDC72D.1010402@canterbury.ac.nz> Leonardo Santagada wrote: > The only clean way to support DSL in python in my view would be to > support something like pyparsing or ANTLR and make it generate a parser > and some simple form of compilation to python bytecodes. How can we > support that? Is this even desired? The way to go is probably to have the parser build an AST. That way you get to reuse the compiler machinery for generating bytecode, without having to worry about the messy details of generating textual Python source. If the DSL->AST transformation is sufficiently context-free, it could probably be specified using some kind of declarative grammar. -- Greg From cliff at develix.com Mon Sep 15 05:14:42 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 20:14:42 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CDC464.4020809@canterbury.ac.nz> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <48CDB3AE.6090704@canterbury.ac.nz> <1221442332.21780.159.camel@portable-evil> <48CDC464.4020809@canterbury.ac.nz> Message-ID: <1221448482.21780.193.camel@portable-evil> On Mon, 2008-09-15 at 14:11 +1200, Greg Ewing wrote: > Cliff Wells wrote: > > > Not that it's terribly relevant to what you say, but: > > > > '-'.join ( > > for J in I: YIELD ( for j in J: YIELD j ) > > ) > > No, I think you were right the first time. The above > looks like it will generate a sequence of iterators, > not a flat sequence of j values. join() takes care of flattening the final yielded iterator. The first one was actually wrong in that it didn't solve the presented problem. > > x = [ J [ a ] > > for a in b > > if c ] > > > > vs > > > > x = for a in b: > > if c: continue J [ a ] > > > > The complexity has barely increased and yet the second form is already > > more readable > > That's a matter of opinion. If you lay out the first one as > > x = [ J [ a ] > for a in b if c ] > > there's not much difference between them. I thought I'd been saying that all along ;-) I think listcomps are only a *clear* win when they are presented in their simplest form, otherwise it really is just preference. Get into nested listcomps and the readability (or more to the point the comprehensibility) pretty much vaporizes. > > Not to mention, the listcomp's placement of the > > yielded result before the loop and condition only works well when it's a > > very simple expression. It doesn't scale well with complexity (not that > > I think it was meant to). > > You're right, it's not meant to. It's meant for the simple > cases where all the syntactic overhead of a full for-statement > and list appending code swamps the content. Given that, adding > any extra syntactic baggage, even just a 'continue' keyword, > reduces its effectiveness for its intended purpose. So then maybe what we could agree on is that there's a place for both? > Also, I think it reads quite nicely with the expression at > the beginning. It's modelled after the mathematical notation > for describing sets: > > {x : some conditions on x} Sure, I think listcomps have a place. I still maintain that they are logically redundant if you have if-expressions (not to mention less flexible), but if we decided they were justifiable syntactic sugar then that's fine too. Cliff From josiah.carlson at gmail.com Mon Sep 15 05:26:15 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 14 Sep 2008 20:26:15 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221425240.21780.102.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> <1221425240.21780.102.camel@portable-evil> Message-ID: On Sun, Sep 14, 2008 at 1:47 PM, Cliff Wells wrote: > On Sun, 2008-09-14 at 13:02 -0700, Cliff Wells wrote: >> On Sun, 2008-09-14 at 12:36 -0700, Josiah Carlson wrote: >> >> > Agreed. For all of the semantic and syntactic gymnastics and >> > discussion about how statements -> expressions would make Python a >> > better language, all I can conclude from the above is "I'm glad Python >> > doesn't do that." > Using Breve as an example again: by your reasoning, the mere existence > of Breve should be reason enough not to support OO in Python (or at the > very least, magic methods). Let me make it clear what I was expressing, because you have no idea. I have personal opinions on what I like and dislike. It became clear to me early on that we disagree on language aesthetics. All I was expressing was that I'm glad that Python didn't look like what you were writing. If you've got a problem with my expressing of that, that's fine, you already decided to move onto a different language. But don't claim to know what I was saying or implying about Python as a language and what it should or should not support. The fact is, Breve is implemented using OO in Python. Could it have been done using a functional approach? Sure, but then you would have had to use multi-calling semantics, closures (which is OO in disguise), or code copy/pasting. Breve is not proof that Python should or should not do something, and to claim otherwise (or believe someone is claiming otherwise) is silly. > Breve more or less a functional DSL that barely resembles Python but > actually *is* Python (in fact, I've seen someone assert that Breve > templates could not possibly be actual Python code). Even you were > initially convinced that I was doing code-generation. Breve abuses I knew that Breve was Python, but I thought it was doing something smart by also doing transparent code generation (because there was discussion about the "speed" of Breve) a'la Cheetah/Spitfire, because I believed that you had worked more on the language than you actually had. I was wrong. But that doesn't make Breve a good example of a DSL, it makes it just another internal DSL using Python. As you say yourself, there are many. > classes and magic methods (albeit in an entirely legal way) to create > the illusion of a declarative DSL. Clearly I think this ability is a > good thing, but it could also be argued that what I've done is > inscrutable and this type of code should not be allowed in Python. Whom has it been argued by? I think the only argument is against adding syntax to make Python purely functional. Using Python syntax and semantics to write interesting features is part of what makes Python great. I've created SQL code generators using __getattr__ and the comparison operators plus __and__ and __or__ for where clauses. And you know what? They looked a million times better than embedding SQL in the code or using stored procedures. None of them would be possible without __magic__ methods. > In fact, your own example of mimicking a dispatch "table" with a class > is arguably object abuse. Personally I don't think it's a bad solution > (especially in the absence of a better way), but you've basically mapped > what is logically a functional problem onto an object. My point is that You didn't even understand my example. I wasn't mapping it into an object, I was using the underlying class creation semantics (specifically metaclasses) to generate a dispatch dictionary. I'll be more clear this time. def dispatcher(name, bases, dict): import __builtin__ dispatch = {} for name, fcn in dict.iteritems(): if callable(name): if '_' in name: typnam, value in name.split('_')[:2] typ = getattr(__builtin__, typnam, None) if not typ: continue dispatch[typ(value)] = fcn else: dispatch[name] = fcn return dispatch If you set the __metaclass__ of a class to the above function, you don't get a class, you get a dictionary. A dispatch dictionary in particular. One that works with arbitrary types (the above only works for builtin types or those that have been inserted into the builtin module). This method shouldn't be new to anyone. I've been doing it since before metaclasses (using functions and locals() instead, because I liked the look better), and I've seen it used in various codebases. You can do similar things to build properties (I have), and any one of a number of other really useful things. > just because something *can* be abused, it isn't reason to throw the > baby out with the proverbial bath water. The more general a tool is, > the more able it is to be used incorrectly (as anyone who's pried a lid > off a can of paint with a screwdriver can attest). Ahh, but the example that I was complaining about *wasn't supposed to be an abuse*, it was supposed to be an example of "how can we make this really simple Python code work with an expression-only version of Python?" That the *simple* Python became so complicated and ugly in the translation to a non-existing expression-only version of Python is unfortunate. But again, I'm glad Python doesn't look like that. Again, don't try to tell me what my reasoning is, I was expressing an opinion on aesthetics. - Josiah From cliff at develix.com Mon Sep 15 05:48:47 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 20:48:47 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CDC72D.1010402@canterbury.ac.nz> References: <1221072180.12242.20.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> <1221425240.21780.102.camel@portable-evil> <48CDBBF1.3000303@canterbury.ac.nz> <98483E4B-7A8A-4C22-9E3D-581C62D604E9@gmail.com> <48CDC72D.1010402@canterbury.ac.nz> Message-ID: <1221450527.21780.199.camel@portable-evil> On Mon, 2008-09-15 at 14:23 +1200, Greg Ewing wrote: > Leonardo Santagada wrote: > > > The only clean way to support DSL in python in my view would be to > > support something like pyparsing or ANTLR and make it generate a parser > > and some simple form of compilation to python bytecodes. How can we > > support that? Is this even desired? > > The way to go is probably to have the parser build an AST. > That way you get to reuse the compiler machinery for generating > bytecode, without having to worry about the messy details > of generating textual Python source. This is the approach Logix took, which is probably why it no longer works under Python 2.5. This is exactly why I find this type of approach unappealing (vs writing an internal DSL). I'm not sure what approach EasyExtend takes, but it's another potential player in this field: http://www.fiber-space.de/EasyExtend/doc/EE.html Regards, Cliff From cliff at develix.com Mon Sep 15 06:18:43 2008 From: cliff at develix.com (Cliff Wells) Date: Sun, 14 Sep 2008 21:18:43 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> <1221425240.21780.102.camel@portable-evil> Message-ID: <1221452323.21780.228.camel@portable-evil> On Sun, 2008-09-14 at 20:26 -0700, Josiah Carlson wrote: > On Sun, Sep 14, 2008 at 1:47 PM, Cliff Wells wrote: > > On Sun, 2008-09-14 at 13:02 -0700, Cliff Wells wrote: > >> On Sun, 2008-09-14 at 12:36 -0700, Josiah Carlson wrote: > >> > >> > Agreed. For all of the semantic and syntactic gymnastics and > >> > discussion about how statements -> expressions would make Python a > >> > better language, all I can conclude from the above is "I'm glad Python > >> > doesn't do that." > > > Using Breve as an example again: by your reasoning, the mere existence > > of Breve should be reason enough not to support OO in Python (or at the > > very least, magic methods). > > Let me make it clear what I was expressing, because you have no idea. > I have personal opinions on what I like and dislike. It became clear > to me early on that we disagree on language aesthetics. All I was > expressing was that I'm glad that Python didn't look like what you > were writing. If you've got a problem with my expressing of that, > that's fine, you already decided to move onto a different language. > But don't claim to know what I was saying or implying about Python as > a language and what it should or should not support. I clearly read more into it than you intended. My apologies. As for opinions on what Python should or shouldn't be, I'll stop having opinions on it when you do (or probably sooner, since it won't matter to me anyway). > The fact is, Breve is implemented using OO in Python. Could it have > been done using a functional approach? Sure, but then you would have > had to use multi-calling semantics, closures (which is OO in > disguise), or code copy/pasting. Breve is not proof that Python > should or should not do something, and to claim otherwise (or believe > someone is claiming otherwise) is silly. I didn't claim Breve proved anything. I resort to it as an example because it's handy for doing so (and because there are so few such examples in Python). > > Breve more or less a functional DSL that barely resembles Python but > > actually *is* Python (in fact, I've seen someone assert that Breve > > templates could not possibly be actual Python code). Even you were > > initially convinced that I was doing code-generation. Breve abuses > > I knew that Breve was Python, but I thought it was doing something > smart by also doing transparent code generation (because there was > discussion about the "speed" of Breve) a'la Cheetah/Spitfire, because > I believed that you had worked more on the language than you actually > had. I was wrong. But that doesn't make Breve a good example of a > DSL, it makes it just another internal DSL using Python. As you say > yourself, there are many. If it did code generation, then it wouldn't be an internal DSL, it would be an external DSL that happened to target the Python language (Cheetah) or VM (Spitfire) as a runtime. That's why I was so careful to mention it several times. > > classes and magic methods (albeit in an entirely legal way) to create > > the illusion of a declarative DSL. Clearly I think this ability is a > > good thing, but it could also be argued that what I've done is > > inscrutable and this type of code should not be allowed in Python. > > Whom has it been argued by? I think the only argument is against > adding syntax to make Python purely functional. I meant hypothetically ("could be argued"). In no way did I say that someone here on this list had taken that stance (it hadn't even come up). > Using Python syntax > and semantics to write interesting features is part of what makes > Python great. I've created SQL code generators using __getattr__ and > the comparison operators plus __and__ and __or__ for where clauses. > And you know what? They looked a million times better than embedding > SQL in the code or using stored procedures. None of them would be > possible without __magic__ methods. > > > In fact, your own example of mimicking a dispatch "table" with a class > > is arguably object abuse. Personally I don't think it's a bad solution > > (especially in the absence of a better way), but you've basically mapped > > what is logically a functional problem onto an object. My point is that > > You didn't even understand my example. I wasn't mapping it into an > object, I was using the underlying class creation semantics > (specifically metaclasses) to generate a dispatch dictionary. I'll be > more clear this time. > > def dispatcher(name, bases, dict): > import __builtin__ > dispatch = {} > for name, fcn in dict.iteritems(): > if callable(name): > if '_' in name: > typnam, value in name.split('_')[:2] > typ = getattr(__builtin__, typnam, None) > if not typ: > continue > dispatch[typ(value)] = fcn > else: > dispatch[name] = fcn > return dispatch > > If you set the __metaclass__ of a class to the above function, you > don't get a class, you get a dictionary. A dispatch dictionary in > particular. One that works with arbitrary types (the above only works > for builtin types or those that have been inserted into the builtin > module). This method shouldn't be new to anyone. I've been doing it > since before metaclasses (using functions and locals() instead, > because I liked the look better), and I've seen it used in various > codebases. You can do similar things to build properties (I have), > and any one of a number of other really useful things. Yep, I didn't get it. I'd assumed you were using __call__ (which would seem a much simpler way to achieve the same thing). Frankly this example only cements my belief that this is overly complicated given the problem to be solved. > > just because something *can* be abused, it isn't reason to throw the > > baby out with the proverbial bath water. The more general a tool is, > > the more able it is to be used incorrectly (as anyone who's pried a lid > > off a can of paint with a screwdriver can attest). > > Ahh, but the example that I was complaining about *wasn't supposed to > be an abuse*, it was supposed to be an example of "how can we make > this really simple Python code work with an expression-only version of > Python?" That the *simple* Python became so complicated and ugly in > the translation to a non-existing expression-only version of Python is > unfortunate. But again, I'm glad Python doesn't look like that. > > Again, don't try to tell me what my reasoning is, I was expressing an > opinion on aesthetics. Again I apologize. But given how many times you did the exact same thing to me here (and rather rudely at that) I think we can call it even. Anyway, I'm done. Regards, Cliff From piotrwysocki at gmail.com Mon Sep 15 12:07:31 2008 From: piotrwysocki at gmail.com (Piotr Wysocki) Date: Mon, 15 Sep 2008 11:07:31 +0100 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> Message-ID: On Sat, Sep 13, 2008 at 7:01 PM, Arnaud Delobelle wrote: > So what does: > > a = (if False: 1) > > evaluate to? Just a note how this problem was solved in Nemerle language: you cannot use 'if' without 'else'. This way you always get a value. If you want an 'if' without 'else', there is a new keyword 'when' (well, there is also 'unless') for the imperative approach. Cheers, Peter From arnodel at googlemail.com Mon Sep 15 21:55:00 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 15 Sep 2008 20:55:00 +0100 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221452323.21780.228.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <1221422567.21780.79.camel@portable-evil> <1221425240.21780.102.camel@portable-evil> <1221452323.21780.228.camel@portable-evil> Message-ID: [snip plenty of wordy arguments about Python, DSLs, misunderstandings, etc, with almost no code, and going nowhere in particular] I'm not a Python guru, I'm not even a professional programmer, worse than that I've never even been a CS student. But I think, humbly, that this thread is much too preoccupied with personal/philosophical opinions about the aesthetics of Python and not enough with *how* to make Python work assuming that statements become expressions. IMHO if we try to work out how, we will come to the conclusion that it would make it a horrible mess. But we can only establish this if we give it a try, and in the process we might learn something useful. And who knows, we might even discover that Python was really meant to be a functional language! After all, this list is called python-ideas. Sorry for the noise. -- Arnaud From greg.ewing at canterbury.ac.nz Tue Sep 16 01:41:22 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 16 Sep 2008 11:41:22 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221448482.21780.193.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <48CDB3AE.6090704@canterbury.ac.nz> <1221442332.21780.159.camel@portable-evil> <48CDC464.4020809@canterbury.ac.nz> <1221448482.21780.193.camel@portable-evil> Message-ID: <48CEF2A2.6030705@canterbury.ac.nz> Cliff Wells wrote: > On Mon, 2008-09-15 at 14:11 +1200, Greg Ewing wrote: >>Cliff Wells wrote: >>> >>>'-'.join ( >>> for J in I: YIELD ( for j in J: YIELD j ) >>>) > > join() takes care of flattening the final yielded iterator. But the final iterator is yielding other iterators, not strings, unless I misunderstand the semantics you have in mind. > Get into nested listcomps and the > readability (or more to the point the comprehensibility) pretty much > vaporizes. That depends on what you mean by "nested listcomps". I agree that nesting one entire listcomp inside another tends to look rather confusing: [f(a) for a in [g(b) for b in y]] But that's not the same thing as having nested *loops* within a single listcomp, which I don't think is particularly bad at all: [f(a, b) for a in x for b in y] or if you prefer, [f(a, b) for a in x for b in y] > Sure, I think listcomps have a place. I still maintain that they are > logically redundant if you have if-expressions (not to mention less > flexible) Yes, but they're also logically redundant even if you don't have statement-expression equivalence, so that's not an argument for merging statements and expressions. -- Greg From greg.ewing at canterbury.ac.nz Tue Sep 16 02:04:04 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 16 Sep 2008 12:04:04 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> Message-ID: <48CEF7F4.8030007@canterbury.ac.nz> Piotr Wysocki wrote: > If you want an 'if' without 'else', there is a new keyword 'when' > (well, there is also 'unless') for the imperative approach. But in a language where every statement is an expression, this just moves the problem to that of what the value of a "when" statement should be. -- Greg From cliff at develix.com Tue Sep 16 02:31:28 2008 From: cliff at develix.com (Cliff Wells) Date: Mon, 15 Sep 2008 17:31:28 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CEF2A2.6030705@canterbury.ac.nz> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <48CDB3AE.6090704@canterbury.ac.nz> <1221442332.21780.159.camel@portable-evil> <48CDC464.4020809@canterbury.ac.nz> <1221448482.21780.193.camel@portable-evil> <48CEF2A2.6030705@canterbury.ac.nz> Message-ID: <1221525088.6222.13.camel@portable-evil> On Tue, 2008-09-16 at 11:41 +1200, Greg Ewing wrote: > Cliff Wells wrote: > > On Mon, 2008-09-15 at 14:11 +1200, Greg Ewing wrote: > >>Cliff Wells wrote: > >>> > >>>'-'.join ( > >>> for J in I: YIELD ( for j in J: YIELD j ) > >>>) > > > > join() takes care of flattening the final yielded iterator. > > But the final iterator is yielding other iterators, > not strings, unless I misunderstand the semantics you > have in mind. I think so. For the simplest case s = 'abc' x = for c in s: YIELD c # x = 'a', 'b', 'c' so for the previous example I = [ 'spam', 'eggs' ] for J in I: # J = 'spam' then 'eggs' YIELD ( # evaluate to an iterable for j in J: YIELD j # j is 's' then 'p' then ... ) so we get '-'.join( 's','p','a','m','e','g','g','s' ) maybe this is clearer as for J in I: tmp = for j in J: YIELD j YIELD tmp > > Get into nested listcomps and the > > readability (or more to the point the comprehensibility) pretty much > > vaporizes. > > That depends on what you mean by "nested listcomps". I agree > that nesting one entire listcomp inside another tends to > look rather confusing: > > [f(a) for a in [g(b) for b in y]] > > But that's not the same thing as having nested *loops* > within a single listcomp, which I don't think is particularly > bad at all: > > [f(a, b) for a in x for b in y] > > or if you prefer, > > [f(a, b) > for a in x > for b in y] > > > Sure, I think listcomps have a place. I still maintain that they are > > logically redundant if you have if-expressions (not to mention less > > flexible) > > Yes, but they're also logically redundant even if you don't > have statement-expression equivalence, so that's not an argument > for merging statements and expressions. Well part of the problem this entire thread has suffered is that there are several related issues being argued simultaneously. Perhaps if I were a better presenter this would have gone differently, but in any case what I was trying to get across is that Python has grown lots of extensions that could be considered redundant and will grow more unless what I see as the seed for these desired extensions is addressed (I believe it to be a desire for better FP support, which merged statements and expressions would largely address). I selected the ternary if-operator, generators and listcomps as examples but I may have overreached a bit. Regards, Cliff From greg.ewing at canterbury.ac.nz Tue Sep 16 03:06:02 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 16 Sep 2008 13:06:02 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221525088.6222.13.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <48CDB3AE.6090704@canterbury.ac.nz> <1221442332.21780.159.camel@portable-evil> <48CDC464.4020809@canterbury.ac.nz> <1221448482.21780.193.camel@portable-evil> <48CEF2A2.6030705@canterbury.ac.nz> <1221525088.6222.13.camel@portable-evil> Message-ID: <48CF067A.7000807@canterbury.ac.nz> Cliff Wells wrote: > s = 'abc' > x = for c in s: YIELD c # x = 'a', 'b', 'c' You mean that the value of (for c in s: YIELD c) is a tuple? Or that it's an iterator that produces that series of values? > I = [ 'spam', 'eggs' ] > > for J in I: # J = 'spam' then 'eggs' > YIELD ( # evaluate to an iterable > for j in J: YIELD j # j is 's' then 'p' then ... > ) > > so we get '-'.join( 's','p','a','m','e','g','g','s' ) I'm still not getting a clear idea of what semantics you intend for a for-loop-with-YIELD. Neither of the interpretations I suggested above (sequence or iterator) seems to produce this result. > maybe this is clearer as > > for J in I: > tmp = for j in J: YIELD j > YIELD tmp No, that's not any clearer. Can you provide a translation into current, valid Python? -- Greg From cliff at develix.com Tue Sep 16 04:00:45 2008 From: cliff at develix.com (Cliff Wells) Date: Mon, 15 Sep 2008 19:00:45 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CF067A.7000807@canterbury.ac.nz> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <48CDB3AE.6090704@canterbury.ac.nz> <1221442332.21780.159.camel@portable-evil> <48CDC464.4020809@canterbury.ac.nz> <1221448482.21780.193.camel@portable-evil> <48CEF2A2.6030705@canterbury.ac.nz> <1221525088.6222.13.camel@portable-evil> <48CF067A.7000807@canterbury.ac.nz> Message-ID: <1221530445.6222.19.camel@portable-evil> On Tue, 2008-09-16 at 13:06 +1200, Greg Ewing wrote: > Cliff Wells wrote: > > > s = 'abc' > > x = for c in s: YIELD c # x = 'a', 'b', 'c' > > You mean that the value of (for c in s: YIELD c) is > a tuple? Or that it's an iterator that produces that > series of values? The latter, although I admit not seeing the importance of the distinction in this case. > > I = [ 'spam', 'eggs' ] > > > > for J in I: # J = 'spam' then 'eggs' > > YIELD ( # evaluate to an iterable > > for j in J: YIELD j # j is 's' then 'p' then ... > > ) > > > > so we get '-'.join( 's','p','a','m','e','g','g','s' ) > > I'm still not getting a clear idea of what semantics > you intend for a for-loop-with-YIELD. Neither of the > interpretations I suggested above (sequence or iterator) > seems to produce this result. > > > maybe this is clearer as > > > > for J in I: > > tmp = for j in J: YIELD j > > YIELD tmp > > No, that's not any clearer. Can you provide a translation > into current, valid Python? My example was a translation of Arnaud's challenge: I = ['spam', 'eggs'] def flatten(I): for J in I: for j in J: yield j >>> '-'.join(flatten(['spam', 'eggs'])) 's-p-a-m-e-g-g-s' It's possible there's a nuance you're seeing that I'm not (and unfortunately we can't test). What are you predicting as the output? Cliff From cliff at develix.com Tue Sep 16 04:30:10 2008 From: cliff at develix.com (Cliff Wells) Date: Mon, 15 Sep 2008 19:30:10 -0700 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <48CF067A.7000807@canterbury.ac.nz> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <48CDB3AE.6090704@canterbury.ac.nz> <1221442332.21780.159.camel@portable-evil> <48CDC464.4020809@canterbury.ac.nz> <1221448482.21780.193.camel@portable-evil> <48CEF2A2.6030705@canterbury.ac.nz> <1221525088.6222.13.camel@portable-evil> <48CF067A.7000807@canterbury.ac.nz> Message-ID: <1221532210.6222.23.camel@portable-evil> On Tue, 2008-09-16 at 13:06 +1200, Greg Ewing wrote: > Cliff Wells wrote: > > > s = 'abc' > > x = for c in s: YIELD c # x = 'a', 'b', 'c' > > You mean that the value of (for c in s: YIELD c) is > a tuple? Or that it's an iterator that produces that > series of values? > > > I = [ 'spam', 'eggs' ] > > > > for J in I: # J = 'spam' then 'eggs' > > YIELD ( # evaluate to an iterable > > for j in J: YIELD j # j is 's' then 'p' then ... > > ) > > > > so we get '-'.join( 's','p','a','m','e','g','g','s' ) > > I'm still not getting a clear idea of what semantics > you intend for a for-loop-with-YIELD. Neither of the > interpretations I suggested above (sequence or iterator) > seems to produce this result. Actually, I think I see the issue. join() is getting a list of iterators (using list notation for simplicity): '-'.join ( [ [ 's','p','a','m'], ['e','g','g','s'] ] ) (I promise you I'm not being intentionally obtuse). Is this the conclusion you were coming to or something else? Cliff From greg.ewing at canterbury.ac.nz Tue Sep 16 07:58:50 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 16 Sep 2008 17:58:50 +1200 Subject: [Python-ideas] Statements vs Expressions... why? In-Reply-To: <1221532210.6222.23.camel@portable-evil> References: <1221072180.12242.20.camel@portable-evil> <1221081763.12242.58.camel@portable-evil> <93A063E1-5FC2-4A15-9BAE-F833750B25D5@googlemail.com> <1221344262.11790.66.camel@portable-evil> <4A9DD6BF-5043-45FF-84DA-EFECC6F8BDD8@googlemail.com> <1221374199.8823.55.camel@portable-evil> <456A8A44-D196-4AD0-B565-EF0B7EF8E283@googlemail.com> <1221380718.21780.13.camel@portable-evil> <1221381415.21780.17.camel@portable-evil> <1221381864.21780.25.camel@portable-evil> <36A568B0-25CE-4721-8761-65FBB53D3C14@googlemail.com> <1221384331.21780.48.camel@portable-evil> <48CDB3AE.6090704@canterbury.ac.nz> <1221442332.21780.159.camel@portable-evil> <48CDC464.4020809@canterbury.ac.nz> <1221448482.21780.193.camel@portable-evil> <48CEF2A2.6030705@canterbury.ac.nz> <1221525088.6222.13.camel@portable-evil> <48CF067A.7000807@canterbury.ac.nz> <1221532210.6222.23.camel@portable-evil> Message-ID: <48CF4B1A.2020505@canterbury.ac.nz> Cliff Wells wrote: > Actually, I think I see the issue. join() is getting a list of > iterators (using list notation for simplicity): > > '-'.join ( [ [ 's','p','a','m'], ['e','g','g','s'] ] ) > > Is this the conclusion you were coming to or something else? Yes, that's right. Arnaud's original code is a single generator containing two nested loops, whereas your version consists of two nested generators. I think your first translation would have produced the right result, because it iterated over each of the inner generators and re-yielded their results in the outer generator. But that's a very expensive way of going about it. -- Greg From cliff at develix.com Thu Sep 18 00:41:27 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 17 Sep 2008 15:41:27 -0700 Subject: [Python-ideas] And now for something completely different Message-ID: <1221691287.6222.94.camel@portable-evil> Okay, for anyone who's still willing to bear with me, I have a completely different approach that might be more palatable (and perhaps even appealing) to some people. I was reading the documentation for Io ( http://www.iolanguage.com ), particulary about how Io implements control structures: in short, it doesn't. What it's able to do is let the user define control structures in Io itself. For example "if" and "for" are merely functions: for ( i, 1, 10, i println ) if ( b == 0, c + 1, d ) Now, the reason this works in Io (and not in Python) is because while you *can* implement a function that mimics the logic of, say, an if-statement, there's no way to implement lazy evaluation of the resulting values. So for instance: def IF ( condition, iftrue, iffalse ): if condition: return iftrue return iffalse result = IF ( hasattr ( spam, 'eggs' ), spam.eggs, 'not found' ) The expression spam.eggs is evaluated regardless of the truthiness of the condition (in fact, causes an exception), so this construct is basically worthless. So... now, here's my shiny new idea: lazy evaluation of function arguments (by some explicit syntactical indicator). Arguments would not be evaluated until they are encountered *at runtime* in the function body. They could be compiled, but not evaluated. You can accomplish this to some degree by using lambda, but I'd much prefer something indicated in the function signature than in the calling code. That is rather than: def foo ( arg ): return arg ( ) foo ( lambda: x ) I'd prefer to write def foo ( lambda: arg ): return arg # evaluation happens here foo ( x ) ("lambda" not necessarily being the actual name of the token in the second case). I think this would have appeal outside of my particular desire (lazy evaluation is a useful feature in-and-of itself) but would also solve my particular issue (I could implement functional versions of any control structure I like, and even implement new ones). Thoughts? Regards, Cliff From bruce at leapyear.org Thu Sep 18 01:18:38 2008 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 17 Sep 2008 16:18:38 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221691287.6222.94.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> Message-ID: On Wed, Sep 17, 2008 at 3:41 PM, Cliff Wells wrote: > I'd prefer to write > > def foo ( lambda: arg ): > return arg # evaluation happens here > foo ( x ) > > ("lambda" not necessarily being the actual name of the token in the > second case). I'd rather use the time machine. You can already do this quite easily. def foo (arg): return arg() # evaluation happens here foo(lambda: x) or def IF(c,t,f=lambda:None): if c: return t() else: return f() IF(hasattr(spam, 'eggs'), lambda: spam.eggs, lambda: 'not found') The idea of something automagically changing from an unevaluated to an evaluated value strikes me as a very bad one so requiring explicit evaluation here is a *good* thing. Aside from that, why should arg be evaluated on return as opposed to being passed along unevaluated until its value is really needed? How would I check to see if an arg was provided? I certainly can't use a default value of None and then check for None since the check would cause it to be evaluated. Maybe we have a special operator that allows us to look at the unevaluated value. What's the context of the evaluation? Can it access variables in the place where it's evaluated? Maybe there's a special way to do that too. Let's not go down that path. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From cliff at develix.com Thu Sep 18 02:27:38 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 17 Sep 2008 17:27:38 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: References: <1221691287.6222.94.camel@portable-evil> Message-ID: <1221697658.6222.112.camel@portable-evil> On Wed, 2008-09-17 at 16:18 -0700, Bruce Leban wrote: > On Wed, Sep 17, 2008 at 3:41 PM, Cliff Wells > wrote: > I'd prefer to write > > def foo ( lambda: arg ): > return arg # evaluation happens here > foo ( x ) > > ("lambda" not necessarily being the actual name of the token > in the > second case). > > > I'd rather use the time machine. You can already do this quite > easily. > > > def foo (arg): > return arg() # evaluation happens here > foo(lambda: x) Except I'd prefer not having lambda sprinkled throughout the calling code (or worse, forgotten). > The idea of something automagically changing from an unevaluated to an > evaluated value strikes me as a very bad one so requiring explicit > evaluation here is a *good* thing. Aside from that, why should arg be > evaluated on return as opposed to being passed along unevaluated until > its value is really needed? Didn't suggest that. That was merely an example. What I'm suggesting is that the *first* time it's used, evaluate then (my example used return for simplicity). > How would I check to see if an arg was provided? I certainly can't > use a default value of None and then check for None since the check > would cause it to be evaluated. I don't consider that a problem. You either care about the value of an argument or you don't. If you care then you will eventually evaluate it anyway. If you don't, then don't evaluate it, ever. > Maybe we have a special operator that allows us to look at the > unevaluated value. What's the context of the evaluation? Can it access > variables in the place where it's evaluated? Maybe there's a special > way to do that too. Let's not go down that path. Agreed we shouldn't go down that path, but don't agree that it's needed. Cliff From cliff at develix.com Thu Sep 18 02:39:00 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 17 Sep 2008 17:39:00 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: References: <1221691287.6222.94.camel@portable-evil> Message-ID: <1221698340.6222.120.camel@portable-evil> On Wed, 2008-09-17 at 16:18 -0700, Bruce Leban wrote: > def IF(c,t,f=lambda:None): > if c: > return t() > else: > return f() > IF(hasattr(spam, 'eggs'), lambda: spam.eggs, lambda: 'not found') > To be clear, the problem is that this leads to brittle code (I want to *enforce* lazy evaluation in a certain context): IF(a<100, b**a, 0) Oops, forgot lambda and nothing crashes but we now constantly evaluate unbounded values. > What's the context of the evaluation? Can it access variables in the > place where it's evaluated? Maybe there's a special way to do that > too. Let's not go down that path. Sorry, missed answering this: the context would be unchanged. It would be evaluated as if it were evaluated at the time of the function call. This may be a serious implementation problem, but I'll let the experts answer that. Regards, Cliff From josiah.carlson at gmail.com Thu Sep 18 02:40:44 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Wed, 17 Sep 2008 17:40:44 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221697658.6222.112.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221697658.6222.112.camel@portable-evil> Message-ID: On Wed, Sep 17, 2008 at 5:27 PM, Cliff Wells wrote: > On Wed, 2008-09-17 at 16:18 -0700, Bruce Leban wrote: >> On Wed, Sep 17, 2008 at 3:41 PM, Cliff Wells >> wrote: >> I'd prefer to write >> >> def foo ( lambda: arg ): >> return arg # evaluation happens here >> foo ( x ) >> >> ("lambda" not necessarily being the actual name of the token >> in the >> second case). >> >> >> I'd rather use the time machine. You can already do this quite >> easily. >> >> >> def foo (arg): >> return arg() # evaluation happens here >> foo(lambda: x) > > Except I'd prefer not having lambda sprinkled throughout the calling > code (or worse, forgotten). > >> The idea of something automagically changing from an unevaluated to an >> evaluated value strikes me as a very bad one so requiring explicit >> evaluation here is a *good* thing. Aside from that, why should arg be >> evaluated on return as opposed to being passed along unevaluated until >> its value is really needed? > > Didn't suggest that. That was merely an example. What I'm suggesting > is that the *first* time it's used, evaluate then (my example used > return for simplicity). > >> How would I check to see if an arg was provided? I certainly can't >> use a default value of None and then check for None since the check >> would cause it to be evaluated. > > I don't consider that a problem. You either care about the value of an > argument or you don't. If you care then you will eventually evaluate it > anyway. If you don't, then don't evaluate it, ever. > >> Maybe we have a special operator that allows us to look at the >> unevaluated value. What's the context of the evaluation? Can it access >> variables in the place where it's evaluated? Maybe there's a special >> way to do that too. Let's not go down that path. > > Agreed we shouldn't go down that path, but don't agree that it's needed. Back when you were talking about statements -> expressions, I was going to point out deferred function calls, but I also ran into the "lambda is ugly" problem. If it weren't for the fact that backticks ` are difficult to distinguish from regular single quotes ', never mind being a PITA to type on some keyboards, I would *almost* suggest that backticks should be used for expressing "a function without arguments, aka a deferred execution expression". But then I'd have to give myself a -1, as it violates TOOWTDI. Then again, the argument could be made that lambdas are just a long spelling of a def without a name, and that ... add = def (arg1, arg2): arg1+arg2 would be better. At that point, we come to def:'deferred' . And I think that's ever worse than backticks. I don't know of a spelling of deferred execution blocks that I would find reasonable :/ . - Josiah From scott+python-ideas at scottdial.com Thu Sep 18 05:05:12 2008 From: scott+python-ideas at scottdial.com (Scott Dial) Date: Wed, 17 Sep 2008 23:05:12 -0400 Subject: [Python-ideas] And now for something completely different In-Reply-To: References: <1221691287.6222.94.camel@portable-evil> <1221697658.6222.112.camel@portable-evil> Message-ID: <48D1C568.5010600@scottdial.com> Josiah Carlson wrote: > I don't know of a spelling of deferred execution blocks that I would > find reasonable :/ . > His request is worse than that. He didn't ask for deferred execution, he asked for deferred values. In python Right Now, you can already do deferred execution blocks (forgetting the "I don't like seeing lambdas!" argument). Rather, he wants to be able to have arbitrary expressions be treated as lazy, not at the call-site either but determined by the receiver: """ I'd prefer to write def foo ( lambda: arg ): return arg # evaluation happens here foo ( x ) """ I'm sorry but this will never, ever, ever happen in Python. That is such a *huge* shift in semantics that you should go use a different language. In such languages, it is *impossible* to know when the expression will actually be evaluated, and while this is fine in a purely functional environment, this is not easy to deal with in imperative programming. If you write "foo(bar())", then you have no way of knowing whether "bar()" is ever run. And in a world that depends on side-effects, that forces you to force evaluations of expressions. You would be forced to write "x = bar(); foo(x)" (assuming assignment isn't lazy), which is less clear than having to write "foo(lambda: bar())" in the opposite case. Explicit is better than implicit. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From josiah.carlson at gmail.com Thu Sep 18 05:26:04 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Wed, 17 Sep 2008 20:26:04 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: <48D1C568.5010600@scottdial.com> References: <1221691287.6222.94.camel@portable-evil> <1221697658.6222.112.camel@portable-evil> <48D1C568.5010600@scottdial.com> Message-ID: On Wed, Sep 17, 2008 at 8:05 PM, Scott Dial wrote: > Josiah Carlson wrote: >> I don't know of a spelling of deferred execution blocks that I would >> find reasonable :/ . >> > > His request is worse than that. He didn't ask for deferred execution, he > asked for deferred values. In python Right Now, you can already do > deferred execution blocks (forgetting the "I don't like seeing lambdas!" > argument). Rather, he wants to be able to have arbitrary expressions be > treated as lazy, not at the call-site either but determined by the receiver: > > """ > I'd prefer to write > > def foo ( lambda: arg ): > return arg # evaluation happens here > foo ( x ) > """ > > I'm sorry but this will never, ever, ever happen in Python. That is such > a *huge* shift in semantics that you should go use a different language. > In such languages, it is *impossible* to know when the expression will > actually be evaluated, and while this is fine in a purely functional > environment, this is not easy to deal with in imperative programming. > > If you write "foo(bar())", then you have no way of knowing whether > "bar()" is ever run. And in a world that depends on side-effects, that > forces you to force evaluations of expressions. You would be forced to > write "x = bar(); foo(x)" (assuming assignment isn't lazy), which is > less clear than having to write "foo(lambda: bar())" in the opposite > case. Explicit is better than implicit. What I was thinking (completely unreasonable, I would be -1 on this) is that the following two lines would be equivalent... x = lambda: foo.bar x = `foo.bar` Sadly, that doesn't really gain you anything - though it would be convenient in the case of an If function (or similar): def If(condition, truev, falsev=`None`): if condition: return truev() return falsev() result = If(x < 5, `foo.smallx(x)`, `foo.largex(x)`) Though, because it's still a function call, it's still slow. And even worse, uses a previously decided-upon ugly pair of characters. While I understand that Cliff wanted auto-execution, I think that requiring a call isn't out of the question (what do Ruby thunks require?) - Josiah From tjreedy at udel.edu Thu Sep 18 05:54:18 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 17 Sep 2008 23:54:18 -0400 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221691287.6222.94.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> Message-ID: Cliff Wells wrote: > You can accomplish this to some degree by using lambda, You can do it completely as far as I know. > but I'd much prefer something indicated in the function signature > than in the calling code. Given Python's nature as a compiled dynamic language, indications are really needed in both caller and callee, as at present: pass a function and call it. Without an explicit indication at the call site, the compiler would have to compile both direct evaluation and function object creation and the code to select between them at runtime according to an attribute of the function resulting from evaluation of the presumed function expression. This would slow *all* function calls, which are slow enough already. Also, one could not tell what objects get passed to a function without knowing the signature in detail, making code harder to read. There is already a way to be flexible without magical arg object switching and with only exceptional need for del/lambda. def iff(cond, if_result, else_result): if cond: if hasattr(if_result, '__call__'): return if_result() else: return if_result else: if hasattr(else_result, '__call__'): return else_result() else: return else_result for i in [0,1,2]: print("I gave you %s gold piece%s." % (i,iff(i==1, '', 's'))) #3.0 #prints (imagine a typical RPG game) I gave you 0 gold pieces. I gave you 1 gold piece. I gave you 2 gold pieces. One only needs the wrapper if the object to be returned is a function example -- iff(cond, lambda: math.sin, lambda: math.cos)(x) or if the expression needs to be guarded by the condition because it would otherwise be invalid (iff(x, lambda x=x: sin(x)/x, 1) -- this might work without the default arg, but might have scope problems) or if it would take a long time. But many conditional values are constant or quick to evaluate and the short-circuiting is not needed. Many of the exceptions might better be defined in functions anyway. def sin_or_cos(cond,x): if cond: return math.sin(x) else: return math.cos(x) def sin_over_x(x): if x: return math.sin(x)/x else: return 0 Terry J. Reedy From talin at acm.org Thu Sep 18 07:09:58 2008 From: talin at acm.org (Talin) Date: Wed, 17 Sep 2008 22:09:58 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221691287.6222.94.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> Message-ID: <48D1E2A6.2080109@acm.org> Cliff Wells wrote: > I was reading the documentation for Io ( http://www.iolanguage.com ), > particulary about how Io implements control structures: in short, it > doesn't. What it's able to do is let the user define control structures > in Io itself. For example "if" and "for" are merely functions: > > for ( i, 1, 10, i println ) > if ( b == 0, c + 1, d ) There's a fundamental problem which is fatal to this idea and others like it (which resulted in the death of many of my own suggestions over the years). The problem is that Python can't support "programmable syntax". Think about the mechanics of import. Unlike C++ or Java, the "import" statement doesn't actually do anything until the program is actually run. The compiler never sees the contents of the imported module. This means that any syntax-changing directives would have to be manually copied into each and every Python source file that used them. (In a sense, this is exactly how import future works.) There's no means to have a standard library of syntax-changing directives. -- Talin From greg.ewing at canterbury.ac.nz Thu Sep 18 07:28:07 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 18 Sep 2008 17:28:07 +1200 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221698340.6222.120.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221698340.6222.120.camel@portable-evil> Message-ID: <48D1E6E7.8070602@canterbury.ac.nz> Cliff Wells wrote: > It would > be evaluated as if it were evaluated at the time of the function call. > This may be a serious implementation problem, but I'll let the experts > answer that. Unless some clever implementation trick could be found, this would have extremely serious performance implications in CPython. Every expression used as a function argument would have to be compiled as a separate function. A closure would need to be created every time it was used, and a Python call executed every time it was evaluated, both fairly expensive operations. Any variables it uses from the surrounding scope would need to be allocated in cells, meaning more memory allocations and access overhead. I could imagine this making the execution of *all* Python code several times slower or worse. -- Greg From cliff at develix.com Thu Sep 18 07:40:01 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 17 Sep 2008 22:40:01 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: References: <1221691287.6222.94.camel@portable-evil> Message-ID: <1221716401.6222.196.camel@portable-evil> On Wed, 2008-09-17 at 23:54 -0400, Terry Reedy wrote: > Cliff Wells wrote: > > You can accomplish this to some degree by using lambda, > > You can do it completely as far as I know. > > > but I'd much prefer something indicated in the function signature > > than in the calling code. > > Given Python's nature as a compiled dynamic language, indications are > really needed in both caller and callee, as at present: pass a function > and call it. Without an explicit indication at the call site, the > compiler would have to compile both direct evaluation and function > object creation and the code to select between them at runtime according > to an attribute of the function resulting from evaluation of the > presumed function expression. This would slow *all* function calls, > which are slow enough already. Yes, I had this suspicion, but I figured I'd wait for an authority to make it plain. However, consider this: foo ( a + b ) # a + b is compiled but not currently evaluated Now, clearly Python would need to know at runtime if foo() accepts a lazy argument prior to calling it, however I'm reminded that Python 3.0 is adding optional static type checking (I believe) which seems to bring the same issue to the table (and possibly at the same expense you mention). Would the mechanism not be more or less the same? Check signature and then apply arguments? Cliff From greg.ewing at canterbury.ac.nz Thu Sep 18 07:36:50 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 18 Sep 2008 17:36:50 +1200 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221716401.6222.196.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221716401.6222.196.camel@portable-evil> Message-ID: <48D1E8F2.70600@canterbury.ac.nz> Cliff Wells wrote: > I'm reminded that Python 3.0 > is adding optional static type checking (I believe) No, it's not. It's adding a way of attaching arbitary annotations to the arguments of a function signature, but Python itself places no interpretation on them. -- Greg From cliff at develix.com Thu Sep 18 07:49:24 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 17 Sep 2008 22:49:24 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: <48D1E8F2.70600@canterbury.ac.nz> References: <1221691287.6222.94.camel@portable-evil> <1221716401.6222.196.camel@portable-evil> <48D1E8F2.70600@canterbury.ac.nz> Message-ID: <1221716964.6222.202.camel@portable-evil> On Thu, 2008-09-18 at 17:36 +1200, Greg Ewing wrote: > Cliff Wells wrote: > > I'm reminded that Python 3.0 > > is adding optional static type checking (I believe) > > No, it's not. It's adding a way of attaching arbitary > annotations to the arguments of a function signature, > but Python itself places no interpretation on them. Ah, so mostly for use by docutils/unittest and the like rather than actual runtime checking. Makes sense. Cliff From cliff at develix.com Thu Sep 18 08:01:59 2008 From: cliff at develix.com (Cliff Wells) Date: Wed, 17 Sep 2008 23:01:59 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: References: <1221691287.6222.94.camel@portable-evil> Message-ID: <1221717719.6222.208.camel@portable-evil> On Wed, 2008-09-17 at 23:54 -0400, Terry Reedy wrote: > Cliff Wells wrote: > > You can accomplish this to some degree by using lambda, > > You can do it completely as far as I know. > > > but I'd much prefer something indicated in the function signature > > than in the calling code. > > Given Python's nature as a compiled dynamic language, indications are > really needed in both caller and callee, as at present: pass a function > and call it. Without an explicit indication at the call site, the > compiler would have to compile both direct evaluation and function > object creation and the code to select between them at runtime according > to an attribute of the function resulting from evaluation of the > presumed function expression. This would slow *all* function calls, > which are slow enough already. Also, one could not tell what objects > get passed to a function without knowing the signature in detail, making > code harder to read. There's one other approach: have two types of functions: and . Whenever Python encounters a declaration of a function with a lazy argument it creates the latter. This would allow Python to know at runtime how to handle it. The difference could be transparent to user code (although it might be unappealing to disguise such a difference). Cliff From piotrwysocki at gmail.com Thu Sep 18 14:19:48 2008 From: piotrwysocki at gmail.com (Piotr Wysocki) Date: Thu, 18 Sep 2008 13:19:48 +0100 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221717719.6222.208.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> Message-ID: On Thu, Sep 18, 2008 at 7:01 AM, Cliff Wells wrote: > There's one other approach: have two types of functions: and > . Whenever Python encounters a declaration of > a function with a lazy argument it creates the latter. This would allow > Python to know at runtime how to handle it. The difference could be > transparent to user code (although it might be unappealing to disguise > such a difference). Why don't you just use strings instead of actual arguments and then just evaluate them if you need them? Someone has already suggested that... As I briefly understand your idea of making Python a more functional language (I like FP myself), I don't think Python is ever going to be one. If you want some FP, just use another language (lazy args will not be implemented ever for sure) Or you can fork Python :) Cheers, Peter From grosser.meister.morti at gmx.net Thu Sep 18 14:22:38 2008 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Thu, 18 Sep 2008 14:22:38 +0200 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221691287.6222.94.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> Message-ID: <48D2480E.2090106@gmx.net> I wonder, why don't you use haskell? Everything is an expression in haskell and everything is lazy. You should be happy with haskell. And there are even haskell compilers, so it's fast, too. ;) -panzi From aahz at pythoncraft.com Thu Sep 18 16:24:11 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 18 Sep 2008 07:24:11 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221691287.6222.94.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> Message-ID: <20080918142411.GA13824@panix.com> On Wed, Sep 17, 2008, Cliff Wells wrote: > > Okay, for anyone who's still willing to bear with me, I have a > completely different approach that might be more palatable (and perhaps > even appealing) to some people. Approach to what? Please keep in mind that this list is archived; using a sensible Subject: line goes a long way toward making that archive useful and usable. (Plus it makes the list more usable in the present because not everyone who reads this list reads every post on it.) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach From tjreedy at udel.edu Thu Sep 18 22:10:40 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 18 Sep 2008 16:10:40 -0400 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221717719.6222.208.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> Message-ID: Cliff Wells wrote: > On Wed, 2008-09-17 at 23:54 -0400, Terry Reedy wrote: >> Cliff Wells wrote: >>> You can accomplish this to some degree by using lambda, >> You can do it completely as far as I know. >> >> > but I'd much prefer something indicated in the function signature >> > than in the calling code. >> >> Given Python's nature as a compiled dynamic language, indications are >> really needed in both caller and callee, as at present: pass a function >> and call it. Without an explicit indication at the call site, the >> compiler would have to compile both direct evaluation and function >> object creation and the code to select between them at runtime according >> to an attribute of the function resulting from evaluation of the >> presumed function expression. This would slow *all* function calls, >> which are slow enough already. Also, one could not tell what objects >> get passed to a function without knowing the signature in detail, making >> code harder to read. > > There's one other approach: have two types of functions: and > . This is what C and languages sort of have with functions and text macros. Without a name convention (such as CPython uses in its codebase, with macros ALL_CAPS, I believe), one cannot tell, for instance, whether f(x) will or will not result in the passing of 'x' or eval(x). It is what some Lisps used to have -- functions and special functions. Users had to memorized which args of which functions were normal and lazy. There must have been some reason why that was superceded. But to repeat: with Python's dynamic name binding, the compiler cannot in general tell whether the 'f' in 'f(x)' will be bound at runtime to a regular or special function, so as I said above, it would have to code for both possibilities. > Whenever Python encounters a declaration of > a function with a lazy argument it creates the latter. This would allow > Python to know at runtime how to handle it. This was already assumed in my original 'select between them at runtime according to an attribute of the function'. > The difference could be > transparent to user code (although it might be unappealing to disguise > such a difference). Unless the argument expression has visible side-effects when evaluated. I think there are also namespace issues that you have missed. Consider def f(`a,`b): # ` indicates lazy evaulation, after Lisp's 'quote ..... blaf + f(a+b,c) -3 When f evaluates a bound to 'a+b', it has to *not* resolve 'b' to f's 'b' but to the caller's 'b'. As someone else said, the unevaluated code for 'a+b' is not enough. A full enclosing function is needed. tjr From cliff at develix.com Fri Sep 19 00:41:41 2008 From: cliff at develix.com (Cliff Wells) Date: Thu, 18 Sep 2008 15:41:41 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> Message-ID: <1221777701.12032.37.camel@portable-evil> On Thu, 2008-09-18 at 16:10 -0400, Terry Reedy wrote: > Cliff Wells wrote: > > Whenever Python encounters a declaration of > > a function with a lazy argument it creates the latter. This would allow > > Python to know at runtime how to handle it. > > This was already assumed in my original 'select between them at runtime > according to an attribute of the function'. Sorry, I must have missed that in reading your reply, but yes both would work. Which of them would be preferable is an implementation detail outside my scope of knowledge. > > The difference could be > > transparent to user code (although it might be unappealing to disguise > > such a difference). > > Unless the argument expression has visible side-effects when evaluated. I think this is the stickiest point (from a philosophical perspective anyway). It's true an explicit side-effect might not happen. However, we already have the inverse of this problem (simple expressions disguising side-effects). Consider: a + b Given that a and be might not be numeric types, what does the above mean? Someone mentioned earlier (sorry, lost the reference) that they found the idea of having to refer to a function signature to unappealing, but magic methods already present a similar problem (disguising potential side-effects). We depend on sane library authors to not abuse this feature. In any case, whenever encountering such a construct we are forced to either assume that the class author did the right thing and the operation makes sense or we refer to the documentation/source to decipher the meaning. I think what I'm suggesting isn't so dissimilar. I will admit that the mere *presence* of a class implies that such side-effects might be present whereas functions only appear ordinary, so there is a tiny bit of fair warning. Still, when encountering something like iff ( cond, iftrue, iffalse ) in a library, I think what I'm suggesting is the least surprising result. It comes down to library and function authors making the intention plain rather than the language providing a specific construct for doing so. I find this acceptable, but others may not. > I think there are also namespace issues that you have missed. Consider > > def f(`a,`b): # ` indicates lazy evaulation, after Lisp's 'quote > ..... > blaf + f(a+b,c) -3 > When f evaluates a bound to 'a+b', it has to *not* resolve 'b' to f's > 'b' but to the caller's 'b'. As someone else said, the unevaluated code > for 'a+b' is not enough. A full enclosing function is needed. Yes, I realize this. I think the overhead of creating a closure would be acceptable provided it only affected this explicitly specified case and not function calls in general. I've long ago accepted that expressive power and efficiency are often mutually exclusive (which I why I moved from C to Python many years ago). Cliff From greg.ewing at canterbury.ac.nz Fri Sep 19 03:14:56 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 19 Sep 2008 13:14:56 +1200 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221777701.12032.37.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> <1221777701.12032.37.camel@portable-evil> Message-ID: <48D2FD10.9060409@canterbury.ac.nz> Cliff Wells wrote: > I think the overhead of creating a closure would > be acceptable provided it only affected this explicitly specified case > and not function calls in general. The problem is telling which cases are affected. If you can't tell until run time, then code has to be generated to handle either possible case for all function arguments, which seems like an effort out of proportion to the problem being solved. And there will still be some runtime overhead in selecting between the two cases. The only way I could see this working is if deferred arguments are marked at *both* the calling and called sites. Then the compiler knows what code to generate, and the called function can check that it has been passed the right kind of argument. -- Greg From carl at carlsensei.com Fri Sep 19 05:51:00 2008 From: carl at carlsensei.com (Carl Johnson) Date: Thu, 18 Sep 2008 17:51:00 -1000 Subject: [Python-ideas] Module/Package-Class Unification Message-ID: <8CCB1C48-3C9B-4DA5-915D-5EE971D1F594@carlsensei.com> Is there a good reason why I can't do something like: import mypackage import mymodule x = mypackage(arg, kwarg) y = mymodule + 3 and other methods, etc. using the __special__ methods. Also, wouldn't it be cool if modules/packages could have metaclass- like handlers? I'm sure there's a good reason why this is just crazy talk implementation-wise, but I'd like to hear what it is. Python has merged types and classes, why not merging in packages and modules, too? -- Carl Johnson From rrr at ronadam.com Fri Sep 19 19:14:31 2008 From: rrr at ronadam.com (Ron Adam) Date: Fri, 19 Sep 2008 12:14:31 -0500 Subject: [Python-ideas] Module/Package-Class Unification In-Reply-To: <8CCB1C48-3C9B-4DA5-915D-5EE971D1F594@carlsensei.com> References: <8CCB1C48-3C9B-4DA5-915D-5EE971D1F594@carlsensei.com> Message-ID: Carl Johnson wrote: > Is there a good reason why I can't do something like: > > import mypackage > import mymodule > > x = mypackage(arg, kwarg) > y = mymodule + 3 > > and other methods, etc. using the __special__ methods. > > Also, wouldn't it be cool if modules/packages could have metaclass-like > handlers? > > I'm sure there's a good reason why this is just crazy talk > implementation-wise, but I'd like to hear what it is. Python has merged > types and classes, why not merging in packages and modules, too? Would these __special__ methods be seperate files like __init__.py is for packages? Ron From josiah.carlson at gmail.com Fri Sep 19 20:37:02 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Fri, 19 Sep 2008 11:37:02 -0700 Subject: [Python-ideas] Module/Package-Class Unification In-Reply-To: References: <8CCB1C48-3C9B-4DA5-915D-5EE971D1F594@carlsensei.com> Message-ID: On Fri, Sep 19, 2008 at 10:14 AM, Ron Adam wrote: > Carl Johnson wrote: >> >> Is there a good reason why I can't do something like: >> >> import mypackage >> import mymodule >> >> x = mypackage(arg, kwarg) >> y = mymodule + 3 >> >> and other methods, etc. using the __special__ methods. >> >> Also, wouldn't it be cool if modules/packages could have metaclass-like >> handlers? >> >> I'm sure there's a good reason why this is just crazy talk >> implementation-wise, but I'd like to hear what it is. Python has merged >> types and classes, why not merging in packages and modules, too? > > > Would these __special__ methods be seperate files like __init__.py is for > packages? I don't believe that was his intent. Generally though, this isn't difficult to do: override __import__. The ultimate question, however, is whether or not this improves code quality, readability, or maintainability. I don't believe it would, so aside from a "purist" approach to unifying, it fails in the face of practicality. - Josiah From cliff at develix.com Fri Sep 19 21:08:05 2008 From: cliff at develix.com (Cliff Wells) Date: Fri, 19 Sep 2008 12:08:05 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: <48D2FD10.9060409@canterbury.ac.nz> References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> <1221777701.12032.37.camel@portable-evil> <48D2FD10.9060409@canterbury.ac.nz> Message-ID: <1221851285.12032.65.camel@portable-evil> On Fri, 2008-09-19 at 13:14 +1200, Greg Ewing wrote: > Cliff Wells wrote: > > > I think the overhead of creating a closure would > > be acceptable provided it only affected this explicitly specified case > > and not function calls in general. > > The problem is telling which cases are affected. If you can't > tell until run time, then code has to be generated to handle > either possible case for all function arguments, which seems > like an effort out of proportion to the problem being solved. You should be able to tell at compile-time which functions are affected (assuming we used a signature indicator. If a function has *any* deferred arguments, it is a special type rather than plain . > And there will still be some runtime overhead in selecting > between the two cases. It depends on how selection is done already (say for functions vs methods). If it's implemented via a type-selected vector then having a separate type then the overhead would be nil. If it requires explicit testing, then there would be at least a small overhead. > The only way I could see this working is if deferred arguments > are marked at *both* the calling and called sites. Then the > compiler knows what code to generate, and the called function > can check that it has been passed the right kind of argument. Actually, that might just be acceptable (if not my preferred form). It meets my desire to enforce lazy evaluation although it retains the ugly lambda in the caller. Cliff From jjb5 at cornell.edu Fri Sep 19 21:46:23 2008 From: jjb5 at cornell.edu (Joel Bender) Date: Fri, 19 Sep 2008 15:46:23 -0400 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221691287.6222.94.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> Message-ID: <48D4018F.2050307@cornell.edu> Cliff Wells wrote: > So... now, here's my shiny new idea: lazy evaluation of function > arguments (by some explicit syntactical indicator). Arguments would not > be evaluated until they are encountered *at runtime* in the function > body. They could be compiled, but not evaluated. This is called "pass by name": Syntactically you need some parameter annotation that says you want pass-by-name semantics, then add a "thunk" layer when those parameters are used in the code. Note that in languages that support pass-by-name, I happen to be familiar with ALGOL, it operates on both the left and right hand side of assignment statements. Perhaps something like this: >>> def foo(arg : __byname__): ... print arg ... arg = (arg * 3) + 1 ... print arg ... >>> x = 2 >>> foo(x) 2 7 >>> print x 7 > I think this would have appeal outside of my particular desire (lazy > evaluation is a useful feature in-and-of itself) but would also solve my > particular issue (I could implement functional versions of any control > structure I like, and even implement new ones). I'm with you, I think there are quite effective uses for this functionality, but nothing that will combat the wave of "why this is fundamentally wrong" or "there's a better way to do it" arguments. Joel From bruce at leapyear.org Fri Sep 19 22:05:13 2008 From: bruce at leapyear.org (Bruce Leban) Date: Fri, 19 Sep 2008 13:05:13 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221851285.12032.65.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> <1221777701.12032.37.camel@portable-evil> <48D2FD10.9060409@canterbury.ac.nz> <1221851285.12032.65.camel@portable-evil> Message-ID: I fail to see any advantage to introducing new syntax and semantics that are likely to confuse people when the capability to do this is already there in a simple and easy to use way. As I said earlier, mark it at the calling site with "lambda:" and at the point where it should be evaluated with "()". And no reasons have been offered for why this should is *so important* that it should be promoted beyond what's already there. Start from the problem, not from the "solution." I don't want to see python become rattlesnake -- where you have to watch carefully every place you step. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From veloso at verylowsodium.com Fri Sep 19 22:06:29 2008 From: veloso at verylowsodium.com (Greg Falcon) Date: Fri, 19 Sep 2008 16:06:29 -0400 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221851285.12032.65.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> <1221777701.12032.37.camel@portable-evil> <48D2FD10.9060409@canterbury.ac.nz> <1221851285.12032.65.camel@portable-evil> Message-ID: <3cdcefb80809191306v20b18e5nd6de47afa1ae99e6@mail.gmail.com> On Fri, Sep 19, 2008 at 3:08 PM, Cliff Wells wrote: > On Fri, 2008-09-19 at 13:14 +1200, Greg Ewing wrote: >> The only way I could see this working is if deferred arguments >> are marked at *both* the calling and called sites. Then the >> compiler knows what code to generate, and the called function >> can check that it has been passed the right kind of argument. > > Actually, that might just be acceptable (if not my preferred form). It > meets my desire to enforce lazy evaluation although it retains the ugly > lambda in the caller. But once you accept that deferred arguments must be marked at the call site, you no longer need any additions to Python. The approach Bruce Leban suggested upthread is sufficient -- wrap your deferred arguments in a lambda, and call to expand them on the function side. This whole discussion, with the non-starter proposal of resurrecting backticks to solve a non-problem, reminds me of an eloquent post earlier this year on the Lua mailing list from Fabien Fleutot: http://lua-users.org/lists/lua-l/2008-02/msg00247.html A part of it applies here almost verbatim. With two changes to reflect Python instead: What drives people so crazy about it? It's not the couple of extra keystrokes, nor the screen real estate eaten by [lambda:a+b]; it's the fact that the current syntax carries a message they strongly disagree with, which says: "anonymous functions are not a lightweight feature to be used pervasively, passing them as function parameters is not the [Python] Way, that's why it looks syntactically odd. If you use them, it ought to be because you do something exceptional, so your code should have a somewhat exceptional appearance". They feel wrong when they use it, and they think they ought to feel good. This thread seems to be about technical solution to the "problem" that you can't really define functions to act as new control structures in Python. But while this may be a design choice you don't agree with, it's not an accident, and it's not a misfeature! Of course you could design a Python-like language that allowed users to create their own control structures. But this would be so different to not be at all Pythonic anymore! The fact that your new language would be a superset of Python is beside the point. Now every large project would have its own control structures defined. Readers of the resulting code will have to spend more time figuring out if the things that look like function calls actually are; reasoning about side effects just got a lot harder. This technical discussion is misguided. There isn't a problem here to fix to begin with. Greg F From cliff at develix.com Fri Sep 19 22:59:31 2008 From: cliff at develix.com (Cliff Wells) Date: Fri, 19 Sep 2008 13:59:31 -0700 Subject: [Python-ideas] And now for something completely different In-Reply-To: <3cdcefb80809191306v20b18e5nd6de47afa1ae99e6@mail.gmail.com> References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> <1221777701.12032.37.camel@portable-evil> <48D2FD10.9060409@canterbury.ac.nz> <1221851285.12032.65.camel@portable-evil> <3cdcefb80809191306v20b18e5nd6de47afa1ae99e6@mail.gmail.com> Message-ID: <1221857971.7130.22.camel@portable-evil> On Fri, 2008-09-19 at 16:06 -0400, Greg Falcon wrote: > This whole discussion, with the non-starter proposal of resurrecting > backticks to solve a non-problem, reminds me of an eloquent post > earlier this year on the Lua mailing list from Fabien Fleutot: > http://lua-users.org/lists/lua-l/2008-02/msg00247.html > A part of it applies here almost verbatim. With two changes to > reflect Python instead: > > What drives people so crazy about it? It's not the couple of extra > keystrokes, nor the screen real estate eaten by [lambda:a+b]; it's > the fact that the current syntax carries a message they strongly > disagree with, which says: "anonymous functions are not a > lightweight feature to be used pervasively, passing them as > function parameters is not the [Python] Way, that's why it looks > syntactically odd. If you use them, it ought to be because you do > something exceptional, so your code should have a somewhat > exceptional appearance". They feel wrong when they use it, and > they think they ought to feel good. This is a pretty compelling argument. The reason this concept works so well in Io is because it's consistent (and is therefore unsurprising) whereas in Python it doesn't fit in the overall scheme of things (it's unexpected). I guess at the end of the day I'm just left with no longer liking the "Python Way". Fair enough. Cliff From tjreedy at udel.edu Sat Sep 20 18:46:06 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 20 Sep 2008 12:46:06 -0400 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221851285.12032.65.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> <1221777701.12032.37.camel@portable-evil> <48D2FD10.9060409@canterbury.ac.nz> <1221851285.12032.65.camel@portable-evil> Message-ID: Cliff Wells wrote: > You should be able to tell at compile-time which functions are affected > (assuming we used a signature indicator. If a function has *any* > deferred arguments, it is a special type > rather than plain . Compiling the function is no problem. Compiling and running the function call is. The class of the object resulting from evaluating an expression is generally not known until runtime. For callable expressions, the only constant expression is a lambda expression. >> And there will still be some runtime overhead in selecting >> between the two cases. > > It depends on how selection is done already (say for functions vs > methods). Python has several callable classes. 'Function' and 'method' (3.0 name) are just two of them. When the compiler sees 'e1(e2)', it assume that the programmer intends e1 to evaluate to a callable and e2 to an argument thereof. So it compiles code to evaluate e1, evaluate e2, and then call e1 with arg e2. The 'selection', such as it is, takes place in the evaluation of the callable expression, by virtue of the class of the resulting object. But the interpreter pays no attention to that, but simply tries to call the presumed callable with the evaluated argument expression by looking for the callable's __call__ method. Which is to say, there is no selection as you seem to mean the term. It is a simple and elegant design, but without major change, not very amenable to your desire for runtime selection of arg processing and passing method. > If it's implemented via a type-selected vector then having> a > separate type then the overhead would > be nil. If it requires explicit testing, then there would be at least a > small overhead. There is neither a vector nor explicit testing. The 'callable' is called, and if it is not callable, the interpreter reports back TypeError: 'whatever' object is not callable Terry Jan Reedy From arnodel at googlemail.com Sat Sep 20 20:35:32 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 20 Sep 2008 19:35:32 +0100 Subject: [Python-ideas] if condition: break idiom Message-ID: Hello, It seems to me that a lot of the time that I use a break statement, it is directly after an if. Typically: while True: do something if condition: break do something else I don't like so much the if .. break which is spread over two lines. Of course I could write while True: do something if condition: break do something else It doesn't read so well either IMHO. I think that this would look better: while True: do something break if condition do something else Now I've had a quick look on a py3k tree (a bit old, rev. 64270): $ grep -r --include "*.py" break py3k/ | wc -l 1418 1418 uses of the break statement in the py3k python code. $ grep -r --include "*.py" -B 1 break py3k/ | grep if | wc -l 680 Of which 680 are immediately preceded by an if statement $ grep -r --include "*.py" "if .*: break" py3k/ | wc -l 107 Of which 107 are preceded by an if on the same line This means that: * 48% of uses of "break" are directly after an "if" * this has been written on one single line about 16% of the time. (I know my greps will include a few false positive but I don't think they are significant :) -- Arnaud From brett at python.org Sat Sep 20 21:07:36 2008 From: brett at python.org (Brett Cannon) Date: Sat, 20 Sep 2008 12:07:36 -0700 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: Message-ID: On Sat, Sep 20, 2008 at 11:35 AM, Arnaud Delobelle wrote: > Hello, > > It seems to me that a lot of the time that I use a break statement, it is > directly after an if. Typically: > > while True: > do something > if condition: > break > do something else > > I don't like so much the if .. break which is spread over two lines. Of > course I could write > > while True: > do something > if condition: break > do something else > > It doesn't read so well either IMHO. I think that this would look better: > > while True: > do something > break if condition > do something else > For some reason this suggestion reminds me of Icon. Anyway, so I take it this suggestion is to extend the break statement to have an 'if' trailer? Or are you trying to make 'break' an expression? If you are after the former, the problem with that is that the special-casing of 'if' might confuse people because they will suddenly want an 'else' clause. Adding that might then cause a naive user to see ``break if x`` and then think 'break' is an expression. Which leads to its own issues. How do you implement 'break' as an expression? It doesn't make much sense to have ``fxn(a, break, b)`` inside a loop. And you can't say 'break' can only appear in the true clause of an 'if' expression as that is too specialized and will lead to potential misuse of the statement (or at least an attempt). I don't see enough benefit from the change to warrant dealing with any of the above issues. -Brett From josiah.carlson at gmail.com Sat Sep 20 21:44:45 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sat, 20 Sep 2008 12:44:45 -0700 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: Message-ID: On Sat, Sep 20, 2008 at 12:07 PM, Brett Cannon wrote: > On Sat, Sep 20, 2008 at 11:35 AM, Arnaud Delobelle > wrote: >> Hello, >> >> It seems to me that a lot of the time that I use a break statement, it is >> directly after an if. Typically: >> >> while True: >> do something >> if condition: >> break >> do something else >> >> I don't like so much the if .. break which is spread over two lines. Of >> course I could write >> >> while True: >> do something >> if condition: break >> do something else >> >> It doesn't read so well either IMHO. I think that this would look better: >> >> while True: >> do something >> break if condition >> do something else >> > > For some reason this suggestion reminds me of Icon. > > Anyway, so I take it this suggestion is to extend the break statement > to have an 'if' trailer? Or are you trying to make 'break' an > expression? > > If you are after the former, the problem with that is that the > special-casing of 'if' might confuse people because they will suddenly > want an 'else' clause. Adding that might then cause a naive user to > see ``break if x`` and then think 'break' is an expression. > > Which leads to its own issues. How do you implement 'break' as an > expression? It doesn't make much sense to have ``fxn(a, break, b)`` > inside a loop. And you can't say 'break' can only appear in the true > clause of an 'if' expression as that is too specialized and will lead > to potential misuse of the statement (or at least an attempt). > > I don't see enough benefit from the change to warrant dealing with any > of the above issues. It gets worse ;) break if condition Also implies... continue if condition Never mind break if condition else continue continue if condition else break Because who would want to write... break if condition continue or continue if condition break But if we can break or continue, why not others? What's wrong with a raise (especially if we surround everything with parens...)? (raise Exception("X")) if condition Never mind assert, yield, throw, return, ... I hope this horse is dead now. - Josiah From arnodel at googlemail.com Sat Sep 20 21:54:59 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 20 Sep 2008 20:54:59 +0100 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: Message-ID: <2D9A1043-61CE-4F01-9C7E-8EBB334304C2@googlemail.com> On 20 Sep 2008, at 20:07, Brett Cannon wrote: > On Sat, Sep 20, 2008 at 11:35 AM, Arnaud Delobelle > wrote: >> >> [...] I think that this would look better: >> >> while True: >> do something >> break if condition >> do something else >> > > For some reason this suggestion reminds me of Icon. > > Anyway, so I take it this suggestion is to extend the break statement > to have an 'if' trailer? Or are you trying to make 'break' an > expression? > No I wasn't suggesting this. > If you are after the former, the problem with that is that the > special-casing of 'if' might confuse people because they will suddenly > want an 'else' clause. Adding that might then cause a naive user to > see ``break if x`` and then think 'break' is an expression. > I thought that the meaning of e.g. break if x < 1 was clear and unambiguous. I don't know if some people do, but I personally have never felt the need to write something like: expression1 if condition else expression2 on its own on a line. So I wouldn't find the above 'break if x < 1' statement confusing :) > Which leads to its own issues. How do you implement 'break' as an > expression? It doesn't make much sense to have ``fxn(a, break, b)`` > inside a loop. And you can't say 'break' can only appear in the true > clause of an 'if' expression as that is too specialized and will lead > to potential misuse of the statement (or at least an attempt). > I certainly wouldn't want 'break' to be an expression. > I don't see enough benefit from the change to warrant dealing with any > of the above issues. > Fair enough! > -Brett -- Arnaud From arnodel at googlemail.com Sat Sep 20 22:26:43 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 20 Sep 2008 21:26:43 +0100 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: Message-ID: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> On 20 Sep 2008, at 20:44, Josiah Carlson wrote: >> On Sat, Sep 20, 2008 at 11:35 AM, Arnaud Delobelle >>> [...] I think that this would look better: >>> >>> while True: >>> do something >>> break if condition >>> do something else >>> >> >> It gets worse ;) > > break if condition > > Also implies... > > continue if condition > That's true, why not? I don't use use continue so much but it seems logical. > Never mind > > break if condition else continue > continue if condition else break > No that would be silly. > Because who would want to write... > break if condition > continue > > or > continue if condition > break > The problem is: when would you need to do this? I pointed out in my original post that almost half of uses of 'break' in the py3k python source come immediately after an 'if'. Now I have never used or seen this: if condition: break continue What would be the point of spelling it 'break if condition else continue'? It would even defeat the idea behind my suggestion, which is to make the structure of a loop more obvious when you scan code: you can spot the 'break if' (and 'continue if'!) statements just by scanning the left hand side of the loop body. > But if we can break or continue, why not others? What's wrong with a > raise (especially if we surround everything with parens...)? > > (raise Exception("X")) if condition > > Never mind assert, yield, throw, return, ... I hope this horse is > dead now. > You have implied one reason why not: raise, assert, yield, throw, return all take an argument which can be an if-expression, so this would make those statements difficult to parse if you are human (and I suspect impossible if you are the Python parser). The fact that neither break nor continue take arguments makes them very 'poor' in the amount of meaning they convey. That's why a lot of people feel like writing the following in one single line. if condition: break Moreover, because we don't have a 'do .. while' construct, 'break' is more important than in other languages to shape loops (especially while loops), so I thought it would be useful to make it easier to spot. The same argument can be made about 'continue' of course. > - Josiah -- Arnaud From carl at carlsensei.com Sun Sep 21 05:41:10 2008 From: carl at carlsensei.com (Carl Johnson) Date: Sat, 20 Sep 2008 17:41:10 -1000 Subject: [Python-ideas] if condition: break idiom Message-ID: Slightly OT, but I think 'break' and 'continue' should be replaced with 'raise Break' and 'raise Continue' in Python 4000, just as we 'raise StopIteration' in generators today. This would be handy, because you could use it in functions called by a list comprehension: def while(x): if x > 10: raise Break else: return x [while(x) for x in range(20)] #produces [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] Right now, you can do something this with list(generator expression) by raising StopIteration, but it is considered a hack, and it doesn't work with list comprehensions. From leif.walsh at gmail.com Sun Sep 21 05:53:58 2008 From: leif.walsh at gmail.com (Leif Walsh) Date: Sat, 20 Sep 2008 23:53:58 -0400 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: Message-ID: On Sat, Sep 20, 2008 at 11:41 PM, Carl Johnson wrote: > def while(x): > if x > 10: > raise Break > else: > return x > > [while(x) for x in range(20)] #produces [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] > > Right now, you can do something this with list(generator expression) by > raising StopIteration, but it is considered a hack, and it doesn't work with > list comprehensions. I imagine what you are thinking of is: """ def While(x): if x > 10: raise StopIteration else: return x list(While(x) for x in range(20)) """ This doesn't seem too horrific to me, and certainly doesn't merit adding Break and Continue as Exception-like objects. I feel this would just make it unbearably hard for beginners moving in from . An alternate construct could be something like: """ def While(x): if x > 10: return None else: return x [w for w in (While(x) for x in range(20)) if w is not None] """ Of course, this looks silly in this example, but in a context where While were to do something nontrivial, I think it would be suitable. -- Cheers, Leif From carl at carlsensei.com Sun Sep 21 06:06:27 2008 From: carl at carlsensei.com (Carl Johnson) Date: Sat, 20 Sep 2008 18:06:27 -1000 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: Message-ID: <7FE47A29-FDA0-46A6-B672-9E8973DF2A93@carlsensei.com> > An alternate construct could be something like: > > """ > def While(x): > if x > 10: > return None > else: > return x > > [w for w in (While(x) for x in range(20)) if w is not None] > """ The problem is that this version won't work if range(20) is replaced with itertools.count() or any other non-finite generator, whereas the raise Break version will. Another nice thing about using raise for loop control is that "raise Continue" can be used to skip elements: def even(x): if x % 2: raise Continue else: return x [even(x) for x in range(10)] # [0, 2, 4, 6, 8] This would have to be broken in two parts to do in current Python (and indeed, that might be the chief advantage of *not* adopting my proposal--TOOWTDI): def even(x): return not x % 2 [x for x in range(10) if even(x)] As for beginners in Python, I don't think learning to use "raise Continue" instead of "continue" will really be that hard. I think the main reason to reject my proposal is that complicated loops shouldn't be done in list comprehensions at all. They should be written out, like reduce expressions. Still, it might be convenient in some cases, and it would be a good chance to cut out unnecessary keywords. -- Carl From rrr at ronadam.com Sun Sep 21 06:09:32 2008 From: rrr at ronadam.com (Ron Adam) Date: Sat, 20 Sep 2008 23:09:32 -0500 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: Message-ID: Carl Johnson wrote: > Slightly OT, but I think 'break' and 'continue' should be replaced with > 'raise Break' and 'raise Continue' in Python 4000, just as we 'raise > StopIteration' in generators today. This would be handy, because you > could use it in functions called by a list comprehension: > > def while(x): > if x > 10: > raise Break > else: > return x > > [while(x) for x in range(20)] #produces [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] > > Right now, you can do something this with list(generator expression) by > raising StopIteration, but it is considered a hack, and it doesn't work > with list comprehensions. Maybe break should raise a StopIteration and continue should raise NextIteration? What would be the effect on performance to for and while loops? Ron From brett at python.org Sun Sep 21 06:34:16 2008 From: brett at python.org (Brett Cannon) Date: Sat, 20 Sep 2008 21:34:16 -0700 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: Message-ID: On Sat, Sep 20, 2008 at 9:09 PM, Ron Adam wrote: > > > Carl Johnson wrote: >> >> Slightly OT, but I think 'break' and 'continue' should be replaced with >> 'raise Break' and 'raise Continue' in Python 4000, just as we 'raise >> StopIteration' in generators today. This would be handy, because you could >> use it in functions called by a list comprehension: >> >> def while(x): >> if x > 10: >> raise Break >> else: >> return x >> >> [while(x) for x in range(20)] #produces [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, >> 10] >> >> Right now, you can do something this with list(generator expression) by >> raising StopIteration, but it is considered a hack, and it doesn't work with >> list comprehensions. > > Maybe break should raise a StopIteration and continue should raise > NextIteration? > > What would be the effect on performance to for and while loops? > You would have to essentially make loops syntactic sugar for try/except statements. And if you had StopIteration and NextIteration you would have to have a nested try/except:: try: LOOP: ... loop stuff ... try: ... loop body ... except NextIteration: pass goto LOOP except StopIteration: pass Exceptions are not expensive, but they are not exactly a no-op either. -Brett From rhamph at gmail.com Sun Sep 21 06:53:01 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 20 Sep 2008 22:53:01 -0600 Subject: [Python-ideas] if condition: break idiom In-Reply-To: <7FE47A29-FDA0-46A6-B672-9E8973DF2A93@carlsensei.com> References: <7FE47A29-FDA0-46A6-B672-9E8973DF2A93@carlsensei.com> Message-ID: On Sat, Sep 20, 2008 at 10:06 PM, Carl Johnson wrote: >> An alternate construct could be something like: >> >> """ >> def While(x): >> if x > 10: >> return None >> else: >> return x >> >> [w for w in (While(x) for x in range(20)) if w is not None] >> """ > > The problem is that this version won't work if range(20) is replaced with > itertools.count() or any other non-finite generator, whereas the raise Break > version will. > > Another nice thing about using raise for loop control is that "raise > Continue" can be used to skip elements: > > def even(x): > if x % 2: > raise Continue > else: > return x > > [even(x) for x in range(10)] # [0, 2, 4, 6, 8] > > This would have to be broken in two parts to do in current Python (and > indeed, that might be the chief advantage of *not* adopting my > proposal--TOOWTDI): def foo(): for x in range(10): if even(x): print "Even!" else: print "Odd!" I would never expect a function could raise an exception which is silently swallowed up. Having break be a statement enforces locality, a good thing IMO. generator expressions may currently swallow up StopIteration, but I argue this is a bug. It may not be worth fixing, but it's a bug nonetheless. The sweet spot for list-comps and generator expressions is when they're *simple*. Once they start to get complicated or fancy (such as making them exit early) you should switch to an explicit for-statement; more lines really is more readable. -- Adam Olsen, aka Rhamphoryncus From greg.ewing at canterbury.ac.nz Sun Sep 21 08:57:19 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 21 Sep 2008 18:57:19 +1200 Subject: [Python-ideas] And now for something completely different In-Reply-To: <1221851285.12032.65.camel@portable-evil> References: <1221691287.6222.94.camel@portable-evil> <1221717719.6222.208.camel@portable-evil> <1221777701.12032.37.camel@portable-evil> <48D2FD10.9060409@canterbury.ac.nz> <1221851285.12032.65.camel@portable-evil> Message-ID: <48D5F04F.50901@canterbury.ac.nz> Cliff Wells wrote: > You should be able to tell at compile-time which functions are affected > (assuming we used a signature indicator. I'm talking about telling at the calling site whether an argument needs to be passed evaluated or unevaluated. You can' tell that at compile time without a marker at the calling site as well. > Actually, that might just be acceptable (if not my preferred form). It > meets my desire to enforce lazy evaluation although it retains the ugly > lambda in the caller. It would really be better to have a new syntax for the calling-site marker, and some kind of flag passed along with the argument behind the scenes. If you just rely on looking at whether the passed argument is a function, you can't tell the difference between a function representing an unevaluated argument and an evaluated argument whose value just happens to be a function. -- Greg From carl at carlsensei.com Sun Sep 21 09:25:48 2008 From: carl at carlsensei.com (Carl Johnson) Date: Sat, 20 Sep 2008 21:25:48 -1000 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: <7FE47A29-FDA0-46A6-B672-9E8973DF2A93@carlsensei.com> Message-ID: Ron Adam wrote: > Maybe break should raise a StopIteration and continue should raise > NextIteration? The trouble with that is that semantically, right now, StopIteration gets caught only during the testing to see if the iteration should go for another loop around, not during the body of the loop. Here are some illustrations of the current behavior: >>> def blah(): ... yield 1 ... raise StopIteration ... yield 2 ... >>> for x in blah(): ... print(x) ... 1 As you can see, it silently swallows the StopIteration raised when evaluating the iterator produced by blah(). >>> def blah(): ... raise StopIteration ... >>> for x in blah(): ... print("hi") ... Traceback (most recent call last): File "", line 1, in File "", line 2, in blah StopIteration Above, the exception isn't swallowed, since blah() hadn't yet been evaluated, since it's a function, not a generator. And if you try to use StopIteration like break, that just won't work: >>> for x in range(2): ... raise StopIteration ... Traceback (most recent call last): File "", line 2, in StopIteration >>> This is why if my proposal were accepted we would need three loop controlling exceptions: StopIteration, to halt the loop while fetching the next value; ContinueIteration, to simulate the behavior of continue; and BreakIteration, to simulate the behavior of break. -- Carl From josiah.carlson at gmail.com Sun Sep 21 18:48:07 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 21 Sep 2008 09:48:07 -0700 Subject: [Python-ideas] if condition: break idiom In-Reply-To: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> Message-ID: On Sat, Sep 20, 2008 at 1:26 PM, Arnaud Delobelle wrote: > > On 20 Sep 2008, at 20:44, Josiah Carlson wrote: > >>> On Sat, Sep 20, 2008 at 11:35 AM, Arnaud Delobelle >>>> >>>> [...] I think that this would look better: >>>> >>>> while True: >>>> do something >>>> break if condition >>>> do something else >>>> >>> >>> It gets worse ;) >> >> break if condition >> >> Also implies... >> >> continue if condition >> > > That's true, why not? I don't use use continue so much but it seems > logical. > >> Never mind >> >> break if condition else continue >> continue if condition else break >> > > No that would be silly. Why? It follows directly from conditional expressions as well as your proposed syntax? >> Because who would want to write... >> break if condition >> continue >> >> or >> continue if condition >> break >> > > The problem is: when would you need to do this? I pointed out in my > original post that almost half of uses of 'break' in the py3k python source > come immediately after an 'if'. Now I have never used or seen this: > > if condition: > break > continue Just because *you've* never seen it, doesn't mean it shouldn't be a valid control structure. while ...: #do some stuff if some_condition: #do some other stuff break if other_condition else continue #do yet some other stuff which is skipped by the line above You are probably thinking; "but Josiah, that secondary clause could be held in an else clause", and I would confirm that. But then I would point out that this one example cannot possibly encompass the entirety of potential uses for the feature. I would then point out that the cases that you are trying to fix, 'break if condition', you inadvertently hide the *flow control* in a mass of words. Break and continue are currently happy little single-word statements. Even when trailing on the same line as an 'if', there is a colon separating them from the if and condition. I would argue that this punctuation is critical for understanding the idea "something conditional is happening here to your flow control". Without that colon, break/continue get to be the only two statements that get an optional 'if' suffix, whose only purpose seems to be to reduce line count. > What would be the point of spelling it 'break if condition else continue'? > It would even defeat the idea behind my suggestion, which is to make the > structure of a loop more obvious when you scan code: you can spot the 'break > if' (and 'continue if'!) statements just by scanning the left hand side of > the loop body. But this doesn't help in that. Break and continue are *already* immediately to the right of whitespace (unless someone tries to make it a single line), and while the *simple* case of... for ... in ...: ... if condition: break ... ... is common, that doesn't mean that the following isn't ... for ... in ...: ... if condition: ... if other_condition: break ... ... Which isn't greatly improved by your suggestion. >> But if we can break or continue, why not others? What's wrong with a >> raise (especially if we surround everything with parens...)? >> >> (raise Exception("X")) if condition >> >> Never mind assert, yield, throw, return, ... I hope this horse is dead >> now. >> > > You have implied one reason why not: raise, assert, yield, throw, return all > take an argument which can be an if-expression, so this would make those > statements difficult to parse if you are human (and I suspect impossible if > you are the Python parser). No, I pointed out a syntax that is unambiguous. How do I know it is unambiguous? Because it's used by the yield *expression* in Python 2.5 and later: x = (yield y) + 1 Though maybe that's a vote against yield, because the rest are statements and don't return anything. > The fact that neither break nor continue take arguments makes them very > 'poor' in the amount of meaning they convey. That's why a lot of people > feel like writing the following in one single line. > > if condition: break It's not a question of amount of meaning, it's a question of context. I've heard explanations at least as much saying that they thought that not having an indented trailer makes the code "prettier", "shorter", "easier to understand" as I've heard the opposite from reviewers arguing against the single-line if idiom. > Moreover, because we don't have a 'do .. while' construct, 'break' is more > important than in other languages to shape loops (especially while loops), > so I thought it would be useful to make it easier to spot. The same > argument can be made about 'continue' of course. Perfectly reasonable, but then the base isn't so much... while 1: ... if condition: break ... it's really... while 1: ... if condition: break Which *has* an idiom in Python. first = 1 while first or condition: first = 0 ... And before you come back with "but that's not as fast", I'll point out that an assignment to a local and an or computation are pretty darn fast. With psyco; even faster: >>> def test1(n): ... i = 0 ... while 1: ... i += 1 ... if i >= n: break ... >>> def test2(n): ... i = 0 ... first = 1 ... while first or n > i: ... first = 0 ... i += 1 ... >>> t = time.time(); test1(10000000); time.time()-t 1.0160000324249268 >>> t = time.time(); test2(10000000); time.time()-t 1.1559998989105225 >>> import psyco >>> psyco.full() >>> t = time.time(); test1(1000000000); time.time()-t 1.4059998989105225 >>> t = time.time(); test2(1000000000); time.time()-t 1.4060001373291016 Worst-case, 15% more time without psyco for the *simplest* possible loop. Less difference as the body of the while increases. With psyco: no difference. But what about, "that's not as pretty as 'break if condition'"? Ok, it's not as pretty. But I would then point out that at least now, you don't have one of those "while 1:" loops that can raise eyebrows. - Josiah From tjreedy at udel.edu Sun Sep 21 20:36:32 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 21 Sep 2008 14:36:32 -0400 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> Message-ID: Josiah Carlson wrote: > >>> break if condition else continue >>> continue if condition else break >>> >> No that would be silly. > > Why? It follows directly from conditional expressions In my view, backwards conditional expressions were Guido's worst mistake. This proposal and comments like the above partly illustrate why I think so. From arnodel at googlemail.com Sun Sep 21 20:50:12 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 21 Sep 2008 19:50:12 +0100 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> Message-ID: On 21 Sep 2008, at 17:48, Josiah Carlson wrote: > On Sat, Sep 20, 2008 at 1:26 PM, Arnaud Delobelle > wrote: >> >> On 20 Sep 2008, at 20:44, Josiah Carlson wrote: >> >>>> On Sat, Sep 20, 2008 at 11:35 AM, Arnaud Delobelle >>>>> >>>>> [...] I think that this would look better: >>>>> >>>>> while True: >>>>> do something >>>>> break if condition >>>>> do something else >>>>> >>>> >>>> It gets worse ;) >>> >>> break if condition >>> >>> Also implies... >>> >>> continue if condition >>> >> >> That's true, why not? I don't use use continue so much but it seems >> logical. >> >>> Never mind >>> >>> break if condition else continue >>> continue if condition else break >>> >> >> No that would be silly. > > Why? It follows directly from conditional expressions as well as your > proposed syntax? > I don't think it does. This has nothing to do with conditional expressions. >>> Because who would want to write... >>> break if condition >>> continue >>> >>> or >>> continue if condition >>> break >>> >> >> The problem is: when would you need to do this? I pointed out in my >> original post that almost half of uses of 'break' in the py3k >> python source >> come immediately after an 'if'. Now I have never used or seen this: >> >> if condition: >> break >> continue > > Just because *you've* never seen it, doesn't mean it shouldn't be a > valid control structure. > What I'm saying is that it doesn't occur often enough to be worried about it. OTOH, 'if condition: break' occurs very commonly. [...] > I would then point out that the cases that you are trying to fix, > 'break if condition', you inadvertently hide the *flow control* in a > mass of words. Break and continue are currently happy little > single-word statements. Even when trailing on the same line as an > 'if', there is a colon separating them from the if and condition. I > would argue that this punctuation is critical for understanding the > idea "something conditional is happening here to your flow control". > Without that colon, break/continue get to be the only two statements > that get an optional 'if' suffix, whose only purpose seems to be to > reduce line count. > Reducing line count was definitely not my aim (see just below) >> What would be the point of spelling it 'break if condition else >> continue'? >> It would even defeat the idea behind my suggestion, which is to >> make the >> structure of a loop more obvious when you scan code: you can spot >> the 'break >> if' (and 'continue if'!) statements just by scanning the left hand >> side of >> the loop body. > > But this doesn't help in that. Break and continue are *already* > immediately to the right of whitespace (unless someone tries to make > it a single line), and while the *simple* case of... > > for ... in ...: > ... > if condition: > break > ... > > ... is common, that doesn't mean that the following isn't ... > You forget: if condition: break > for ... in ...: > ... > if condition: > ... > if other_condition: > break > ... > ... > > Which isn't greatly improved by your suggestion. There's some truth in this. >>> But if we can break or continue, why not others? What's wrong >>> with a >>> raise (especially if we surround everything with parens...)? >>> >>> (raise Exception("X")) if condition >>> >>> Never mind assert, yield, throw, return, ... I hope this horse is >>> dead >>> now. >>> >> >> You have implied one reason why not: raise, assert, yield, throw, >> return all >> take an argument which can be an if-expression, so this would make >> those >> statements difficult to parse if you are human (and I suspect >> impossible if >> you are the Python parser). > > No, I pointed out a syntax that is unambiguous. How do I know it is > unambiguous? Because it's used by the yield *expression* in Python > 2.5 and later: > > x = (yield y) + 1 > > Though maybe that's a vote against yield, because the rest are > statements and don't return anything. > Yes, they're statements, I wouldn't want them to be surrounded by parentheses. I only proposed "break if" (and "continue if", perhaps), I don't see why every statement should be equipped with a trailing "if" as a necessary consequence. >> The fact that neither break nor continue take arguments makes them >> very >> 'poor' in the amount of meaning they convey. That's why a lot of >> people >> feel like writing the following in one single line. >> >> if condition: break > > It's not a question of amount of meaning, it's a question of context. > I've heard explanations at least as much saying that they thought that > not having an indented trailer makes the code "prettier", "shorter", > "easier to understand" as I've heard the opposite from reviewers > arguing against the single-line if idiom. > if condition: break on a single line happens a lot, whether liked by reviewers or not. >> Moreover, because we don't have a 'do .. while' construct, 'break' >> is more >> important than in other languages to shape loops (especially while >> loops), >> so I thought it would be useful to make it easier to spot. The same >> argument can be made about 'continue' of course. > > Perfectly reasonable, but then the base isn't so much... > > while 1: > ... > if condition: break > ... > > it's really... > > while 1: > ... > if condition: break > I don't agree with that: the absence of do .. while liberates the loop construct in python from its constraints and the first form above becomes more common. > Which *has* an idiom in Python. > > first = 1 > while first or condition: > first = 0 > ... > I would not use this, not because it is slower, but because it is uglier. > And before you come back with "but that's not as fast", I'll point out > that an assignment to a local and an or computation are pretty darn > fast. With psyco; even faster: > >>>> def test1(n): > ... i = 0 > ... while 1: > ... i += 1 > ... if i >= n: break > ... >>>> def test2(n): > ... i = 0 > ... first = 1 > ... while first or n > i: > ... first = 0 > ... i += 1 > ... >>>> t = time.time(); test1(10000000); time.time()-t > 1.0160000324249268 >>>> t = time.time(); test2(10000000); time.time()-t > 1.1559998989105225 >>>> import psyco >>>> psyco.full() >>>> t = time.time(); test1(1000000000); time.time()-t > 1.4059998989105225 >>>> t = time.time(); test2(1000000000); time.time()-t > 1.4060001373291016 > My tests: >>> t1=timeit.Timer("test1(10000000)", "from loops import test1").timeit(5) >>> t2=timeit.Timer("test2(10000000)", "from loops import test2").timeit(5) >>> t1, t2 (5.3120660781860352, 7.4985768795013428) >>> t2/t1 1.4116121240084343 test1 is 41% faster, that is not negligible. I don't think the fact that psyco optimises the code is a valid argument. > Worst-case, 15% more time without psyco for the *simplest* possible > loop. Less difference as the body of the while increases. With > psyco: no difference. > > But what about, "that's not as pretty as 'break if condition'"? Ok, > it's not as pretty. But I would then point out that at least now, you > don't have one of those "while 1:" loops that can raise eyebrows. > Well, they don't raise my eyebrows :) > - Josiah Anyway, there hasn't been a flurry of positive responses so far so I don't think this is going to go much further than this reply... -- Arnaud From josiah.carlson at gmail.com Sun Sep 21 20:52:05 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 21 Sep 2008 11:52:05 -0700 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> Message-ID: On Sun, Sep 21, 2008 at 11:36 AM, Terry Reedy wrote: > Josiah Carlson wrote: >> >>>> break if condition else continue >>>> continue if condition else break >>>> >>> No that would be silly. >> >> Why? It follows directly from conditional expressions > > In my view, backwards conditional expressions were Guido's worst mistake. > This proposal and comments like the above partly illustrate why I think so. I don't know if it was his worst mistake in language design, but I agree it did open the door for such proposals. - Josiah From arnodel at googlemail.com Sun Sep 21 21:28:00 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 21 Sep 2008 20:28:00 +0100 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> Message-ID: On 21 Sep 2008, at 19:36, Terry Reedy wrote: > Josiah Carlson wrote: >> >>>> break if condition else continue >>>> continue if condition else break >>>> >>> No that would be silly. >> Why? It follows directly from conditional expressions > > In my view, backwards conditional expressions were Guido's worst > mistake. This proposal and comments like the above partly > illustrate why I think so. > I don't know what was the motivation for including conditional expression in the language, but: condition and x or y is better replaced with: x if condition else y So from a pragmatic point of view, if-expressions work as some kind of "vaccine" against the above. -- Arnaud From arnodel at googlemail.com Sun Sep 21 21:52:57 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 21 Sep 2008 20:52:57 +0100 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> Message-ID: On 21 Sep 2008, at 20:24, Josiah Carlson wrote: [...] > Not all statements; all *control flow* statements. Return, yield, > continue, break, assert, ..., can all change program flow. To say > that break and continue should be special cased is silly, as "Special > cases aren't special enough to break the rules". As such, why > continue and break but not return or yield? Further, the syntax is > *so very similar* to conditional expressions if vs. if > else , the lack of an else clause could be confusing to those who > have seen and used conditional expressions before, never mind the > meaning of them. Well my view was that break and continue are the only two statements that relate to loops. [...] >> I don't agree with that: the absence of do .. while liberates the >> loop >> construct in python from its constraints and the first form above >> becomes >> more common. > > But you were just arguing that the *lack* of do/while makes the > embedded if necessary. Now you are saying that it *liberates* us from > the control-flow induced by do/while. ;) There's an argument that > says rather than treat the symptom (breaks in the body), treat the > disease (lack of a do/while). But since the lack of a do/while isn't > a disease, by your own words, then the symptom is not a bug, it's a > feature ;) > There is a missing link in your interpretation of my argumentation. It is that I haved noticed that, as I do not have a do .. while construct at my disposal in Python, I do not try to shape my loops into this structure anymore. I almost *never* write: while True: ... if condition: break But most of the time it seems that the correct structure for a loop comes as while True: ... if condition: break ... In fact, I would be happy with getting rid of the while loop and replacing it with a simple 'loop' constuct, where: while condition: ... would be written as: loop: if condition: break ... However I see this as too radical to propose :) >>> Which *has* an idiom in Python. >>> >>> first = 1 >>> while first or condition: >>> first = 0 >>> ... >>> >> >> I would not use this, not because it is slower, but because it is >> uglier. > > That's funny, because whenever I use it in a production codebase, > coworkers always comment how they like it because it pushes the > condition for the loop at the loop header rather than embedding it in > the (sometimes long) body. In particular, I've seen the lack of a > do-while in Python result in an if clause at the bottom of a 150 line > while, which had been confusing as hell for anyone who got to touch > that code. Moving it up to the top made it clear and resulted in a > removal of half a dozen lines of comments at the top explaining why > the loop was constructed like this. After the above translation, > those comments became "this emulates a do-while clause". > This makes a lot of sense, if you use a do .. while concept in your loops. Now I feel bad that I used the word "uglier" ... > >> Anyway, there hasn't been a flurry of positive responses so far so >> I don't >> think this is going to go much further than this reply... > > Syntax changes are tough to get agreement on in Python. > Thank you for your comments, they all make a lot of sense. -- Arnaud From josiah.carlson at gmail.com Sun Sep 21 22:24:10 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 21 Sep 2008 13:24:10 -0700 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> Message-ID: On Sun, Sep 21, 2008 at 12:52 PM, Arnaud Delobelle wrote: > On 21 Sep 2008, at 20:24, Josiah Carlson wrote: >> Not all statements; all *control flow* statements. Return, yield, >> continue, break, assert, ..., can all change program flow. To say >> that break and continue should be special cased is silly, as "Special >> cases aren't special enough to break the rules". As such, why >> continue and break but not return or yield? Further, the syntax is >> *so very similar* to conditional expressions if vs. if >> else , the lack of an else clause could be confusing to those who >> have seen and used conditional expressions before, never mind the >> meaning of them. > > Well my view was that break and continue are the only two statements that > relate to loops. > [...] Superficially yes, but the execution of raise, return or yield can result in the loop never completing another pass. That a loop can occur in an except clause even offers the ability for the no-argument raise to make sense even there ;) . >>> I don't agree with that: the absence of do .. while liberates the loop >>> construct in python from its constraints and the first form above becomes >>> more common. >> >> But you were just arguing that the *lack* of do/while makes the >> embedded if necessary. Now you are saying that it *liberates* us from >> the control-flow induced by do/while. ;) There's an argument that >> says rather than treat the symptom (breaks in the body), treat the >> disease (lack of a do/while). But since the lack of a do/while isn't >> a disease, by your own words, then the symptom is not a bug, it's a >> feature ;) >> > > There is a missing link in your interpretation of my argumentation. It is > that I haved noticed that, as I do not have a do .. while construct at my > disposal in Python, I do not try to shape my loops into this structure > anymore. I almost *never* write: > > while True: > ... > if condition: break > > But most of the time it seems that the correct structure for a loop comes as > > while True: > ... > if condition: break > ... That also has a translation with the "first" idiom, though it's a bit uglier (no, really): first = 1 while first or condition: if not first: ... first = 0 ... Really though, what is pretty/elegant really varies depending on what those ... turn into. To be honest, I've only ever used the above once or twice, though the standard first idiom I've used dozens of times. Though if you really want to have fun, there's nothing like exploiting while loops for checking error conditions. ;) def foo(...): first = 1 while first: first = 0 if error_condition1: break ... #other error conditions, processing, whatever else: #only executes if everything is ok #error condition - Josiah From arnodel at googlemail.com Sun Sep 21 23:10:39 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 21 Sep 2008 22:10:39 +0100 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> Message-ID: <3DB36F0C-EA6A-4D6F-9513-A23250750DC4@googlemail.com> On 21 Sep 2008, at 21:24, Josiah Carlson wrote: [...] > Really though, what is pretty/elegant really varies depending on what > those ... turn into. To be honest, I've only ever used the above once > or twice, though the standard first idiom I've used dozens of times. > Though if you really want to have fun, there's nothing like exploiting > while loops for checking error conditions. ;) > > def foo(...): > first = 1 > while first: > first = 0 > if error_condition1: > break > ... #other error conditions, processing, whatever > else: > #only executes if everything is ok > #error condition > Why not write: def foo(...): while True: if error_condition1: break # Other error conditions, processing, etc # only executes if everything is OK # Error condition Is there something that I don't see? -- Arnaud From arnodel at googlemail.com Mon Sep 22 00:13:40 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 21 Sep 2008 23:13:40 +0100 Subject: [Python-ideas] if condition: break idiom In-Reply-To: <3DB36F0C-EA6A-4D6F-9513-A23250750DC4@googlemail.com> References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> <3DB36F0C-EA6A-4D6F-9513-A23250750DC4@googlemail.com> Message-ID: <2FCF19EA-B6E3-44C4-AEC9-03335D989527@googlemail.com> There's a missing 'return' in my last post! On 21 Sep 2008, at 22:10, Arnaud Delobelle wrote: > def foo(...): > while True: > if error_condition1: > break > # Other error conditions, processing, etc > # only executes if everything is OK return > > # Error condition -- Arnaud From josiah.carlson at gmail.com Mon Sep 22 03:21:31 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 21 Sep 2008 18:21:31 -0700 Subject: [Python-ideas] if condition: break idiom In-Reply-To: <2FCF19EA-B6E3-44C4-AEC9-03335D989527@googlemail.com> References: <5C7627CB-9434-4A77-8B4B-EBC1C588D4FD@googlemail.com> <3DB36F0C-EA6A-4D6F-9513-A23250750DC4@googlemail.com> <2FCF19EA-B6E3-44C4-AEC9-03335D989527@googlemail.com> Message-ID: On Sun, Sep 21, 2008 at 3:13 PM, Arnaud Delobelle wrote: > There's a missing 'return' in my last post! > > On 21 Sep 2008, at 22:10, Arnaud Delobelle wrote: > >> def foo(...): >> while True: >> if error_condition1: >> break >> # Other error conditions, processing, etc >> # only executes if everything is OK > > return >> >> # Error condition Short answer: there are many variants, not all of them can be translated into the above. - Josiah From and-dev at doxdesk.com Thu Sep 25 15:34:05 2008 From: and-dev at doxdesk.com (Andrew Clover) Date: Thu, 25 Sep 2008 15:34:05 +0200 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: Message-ID: <48DB934D.8030808@doxdesk.com> Arnaud Delobelle wrote: > while True: > do something > if condition: break > do something else I think mid-test/multi-test loops are common enough that it might be worth explicit syntax to pick out the loop exit points: while True: do something andwhile not condition: do something else (or 'and while' if one wanted to save on reserved words. 'not condition' because the logic is reversed from the quoted example.) Similarly for 'if', I often find myself with unsatisfactory constructs like: if condition: do something if othercondition: success else: failure else: failure I would like to flatten together the else clauses: if condition: do something andif othercondition: success else: failure From josiah.carlson at gmail.com Thu Sep 25 18:54:18 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Thu, 25 Sep 2008 09:54:18 -0700 Subject: [Python-ideas] if condition: break idiom In-Reply-To: <48DB934D.8030808@doxdesk.com> References: <48DB934D.8030808@doxdesk.com> Message-ID: On Thu, Sep 25, 2008 at 6:34 AM, Andrew Clover wrote: > Arnaud Delobelle wrote: > >> while True: >> do something >> if condition: break >> do something else > > I think mid-test/multi-test loops are common enough that it might be worth > explicit syntax to pick out the loop exit points: > > while True: > do something > andwhile not condition: > do something else > That breaks the flow of the loop, and it is not visually clear that "do something else" is a part of the while loop. This is one of many reasons why switch/case statements failed. > (or 'and while' if one wanted to save on reserved words. 'not condition' > because the logic is reversed from the quoted example.) > > Similarly for 'if', I often find myself with unsatisfactory constructs like: > > if condition: > do something > if othercondition: > success > else: > failure > else: > failure > > I would like to flatten together the else clauses: > > if condition: > do something > andif othercondition: > success > else: > failure That becomes ambiguous in the case of more than 2 levels of if, never mind needing andelif, never mind being wholly unreadable by non-english speakers. Flat is better than nested, but nested indents do offer clarity about what code does. A brief skimming of the above tells me nothing about the control flow of the code. - Josiah From jim_hill_au-24 at yahoo.com.au Fri Sep 26 11:03:10 2008 From: jim_hill_au-24 at yahoo.com.au (Jim Hill) Date: Fri, 26 Sep 2008 19:03:10 +1000 Subject: [Python-ideas] if condition: break idiom In-Reply-To: References: <48DB934D.8030808@doxdesk.com> Message-ID: <48DCA54E.7010403@yahoo.com.au> Josiah Carlson wrote: > On Thu, Sep 25, 2008 at 6:34 AM, Andrew Clover wrote: >> Arnaud Delobelle wrote: >> >> I think mid-test/multi-test loops are common enough that it might be worth >> explicit syntax to pick out the loop exit points: >> >> while True: >> do something >> andwhile not condition: >> do something else >> > > That breaks the flow of the loop, and it is not visually clear that > "do something else" is a part of the while loop. while True: do something *breakif condition do something else *continueif condition do some other thing I don't know if you can start a key word with * but it does pick out the loop exit points without complicating the indentation. Is it terribly bad? It was rejected a few months ago, but possibly because it was lumped in with some other stuff. The only alternative I can think of is syntax colouring: recommend that keywords that break the flow should be treated differently from other keywords in Python code editors. eg break, continue, return, exit in bold red. -- jh