On Fri, 2008-09-12 at 09:51 -0700, Josiah Carlson wrote:

On Fri, Sep 12, 2008 at 8:55 AM, Cliff Wells <cliff@develix.com> wrote:

...

Agreed, and I had intended to move to that stage once I felt there was a consensus that there is actually something that needs solving. What I failed to make clear in my position is that I'm not trying to add a feature for a it's own sake, rather that the current process for limiting the growth of the Python language is doomed to failure.

The growth limiting mechanism is doomed to failure, or the language is doomed to failure?

I'm asserting that the limiting mechanism is doomed to failure.

...

I'm going to try to rephrase the fundamental problem I see here and hopefully I won't fail so dismally this time:

1) I assert that small is better than large when it comes to limiting complexity.

It would seem that you are saying that limiting Python's complexity is a bad thing. Please correct me if my understanding of your statement is incorrect.

No, I'm saying limiting complexity is good and that smaller is better.

I'm personally of the opinion that limiting the complexity of Python as a language is a good thing. And so far, I think Guido has done a fairly very good job at keeping Python moving in a positive direction.

Yes, but I also feel that the language has already grown to a point where only a shrinking subset of Python users fully know and understand the full feature set. For people who have been using Python for many years, they've possibly had time to digest the new features. I don't believe new users fare as well.

...

2) The primary method of limiting Python's core feature growth is for the BDFL to dig in his heels and reject ideas until they are adequately shown to be broadly needed and implementable.

3) This method has failed in the past*. In fact, I assert that this method is guaranteed to fail unless all ideas that add syntactical structures are rejected outright, regardless of their utility. It cannot limit the growth of Python's core, it can only limit the rate at which it grows.

If the rate of language feature growth is slowed by Guido, then obviously the growth of Python's core language is limited.

A slow boat still reaches its destination. My position is that by obviating the need for a large class of syntax extensions, the onus is moved from the Python core team onto the user desiring a particular extension and by doing so, things like community pressure to add X can be reduced and by extension, the likelihood of the process folding under that pressure becomes much less likely.

I would argue that this is a good thing, as the more language syntax and features that are present, the steeper the learning curve for new users of the language.

Absolutely. But this isn't what Python has been doing (as you acknowledge below). We seem to be agreeing on this point, so I believe our disagreement must center around whether the current process continues to be sufficient.

Python 2.5.2 is a wholly different beast from the Python 1.5.2 that I started with, but I've had nearly 10 years to update my understanding of the language. For new users? Metaclasses, list comprehensions, generator expressions, generators, ... I get asked about them all. And the language keeps growing!

I switched from C to Python right as 1.5.1 was being phased out. To this day I continue to use a subset of Python that is probably much closer to 1.5.x than 2.5. This may be attributed to a personal failing (my self-professed laziness), but the fact remains that I prefer a small, versatile set of constructs to a large set of specialized ones. C, for all of its shortcomings, had one absolutely compelling feature: you could keep the entire language in your head and you could do it in an amazingly short time. Python 1.5 was similar in scope. 2.5 is arguably at or beyond that limit for most people.

...

4) As Python moves into new domains, as new programming languages come into vogue, and as the art of programming itself advances (albeit glacially), users will perceive needs in Python and will clamor for extensions. Some of these will eventually be accepted.

But not all. Why? Because not all proposed extensions to Python are worth discussing outside of a water-cooler "wouldn't it be neat if?" discussion. See the proposed extension to list comprehensions where "order by" was proposed from sql (a year or two ago at this point). That shouldn't have even made it to python-dev. Yet it did. Why? Because everyone has their "I really want X", and too few people are willing to acknowledge that they can be wrong.

Right, but I am suggesting that perhaps at least some of this problem (the need users feel for extensions) is currently built into the language. I also made this point to bolster my argument that feature-creep is inevitable. Outside of the object unification that's been happening over the last few years, most of the features being added to Python appear to be oriented to supporting the FP paradigm (generators, list comps, ternary if, etc). However these features appear to be added in an ad hoc fashion. I'm not suggesting they weren't thought out, only that they were added based on demand rather than as part of an overall plan as the object unification was. It appears to me that most of these "water-cooler" suggestions are proposed precisely because a) Users encounter a situation where an elegant solution isn't obvious and they feel they must resort to magic and hackery. b) The elegant solution they were looking for is embodied as an extension or requires knowledge that is more advanced than the problem at hand would seem to require (e.g. metaclasses). c) The PEPs (or more commonly, the flamewars on c.l.py) begin. For example, earlier in this thread you provided an elegant solution to what appeared (to myself and at least one other person) to be an intractable problem. But of course, it wasn't the natural solution to most people. It's ironic, because I've actually used a similar solution before, but wasn't able to see it this time. In fact, every time I have to use it, I have to reinvent it because it isn't the natural way the problem is expressed. The problem is most naturally expressed procedurally but must be solved with an OO paradigm.

...

5) Given the added desire to maintain backwards-compatibility, old features will not be shed as fast as new ones are added (unless that becomes part of the process, which doesn't seem practicable to me).

Python 3.0 is one of the ways that features are shed. While Guido has stated that he doesn't believe that there will be a Python 4k (aka Python 4.0) that hits the reset button again, I think that there needs to be one, so that Python *can* get rid of bad features that we were wrong about. There's also the PEP process, which as long as there isn't sweeping "break everything" changes, it would seem that we can remove features as time goes on.

Sure, and I think this is a valuable process.

...

6) I believe that a large class of these features could be rendered moot or redundant if the language embraced a more sweeping and fundamental change. While this won't absolutely prevent the language's growth, it provides a built-in deterrent.

...To a functional programming paradigm. That's what you meant to say, right?

Yes, although by "to" I would mean "allow for" rather than "convert to". In other words, not convert to or even encourage the FP paradigm, but also not prevent it. It would be embracing multi-paradigm approaches rather than abandoning one in favor of another. Python has always been multi-paradigm to an extent (procedural and object), but has always been somewhat adverse to the functional paradigm (aside from a few helpful things). Before Python's object unification efforts, many OO purists declaimed Python because its hybrid approach. Pythonistas claimed that they were wrong because "practicality beats purity", etc. But of course, we've all seen that to a large degree they were right. Practicality might beat purity, but practicality plus purity trumps both. Functional programming is, in fact, simply the logical extension of procedural programming (or more precisely, procedural is a subset of functional). They are in no way mutually exclusive. Much as Python's OO was hampered by distinctions between builtin classes and user-defined classes (amongst other things), so is Python's procedural support hampered by it's rejection of the functional paradigm.

...

There's an old quote by Larry Wall: "Perl is ugly because people wanted it that way". Perl took the same approach to limiting features that Python does, with the notable difference that Larry didn't reject ideas as consistently as Guido (or apparently at all). Nevertheless, we're now simply discussing the *rate* at which the language becomes large and inconsistent rather than whether it will or not.

If we agree that the issues I outline above are valid, then I think we can start bickering over possible solutions and how those solutions would affect Python on the whole.

I don't agree.

Well, that's a start. Regards, Cliff

On Fri, 2008-09-12 at 14:24 -0600, Adam Olsen wrote:

On Fri, Sep 12, 2008 at 1:29 PM, Cliff Wells <cliff@develix.com> wrote:

...

A slow boat still reaches its destination. My position is that by obviating the need for a large class of syntax extensions, the onus is moved from the Python core team onto the user desiring a particular extension and by doing so, things like community pressure to add X can be reduced and by extension, the likelihood of the process folding under that pressure becomes much less likely.

Again, a vast difference in what we consider complexity.

Well there's two forms of complexity to consider: language complexity and user-program complexity. A language that's too simple (e.g. assembler or C) can lead to an explosion of complexity when actually used to develop a non-trivial program. I'd actually agree that language features help reduce program complexity. For instance even the simple case of Python lacking the equivalent of repeat..until or do..while loop forces a tiny bit of complexity (in the form of a couple extra lines of code and usually an extra exit point from a loop) into a user's programs. The complexity I'm referring to is the fact that Python's collection of statements are rapidly forking into two forms: the original statement form, and newer expression forms. Worse, in some cases these expression forms exist in several sub-forms (e.g. list comps, expression comps, generators, etc). These forms all have a single common *logical* operation in mind (looping), but mutate into other forms because they are too specialized. It's also worth noting that most of them use an inverted syntax from the original statement in order to help distinguish them. I'm claiming that a single more flexible construct could potentially be used in place of all of them (for-expression with yield), and further, I'm claiming this is a case where simplifying the language would not complicate the user's program and in many cases might help to simplify it.

For a user, does it matter that we use "len(foo)" rather than "len foo"? A little bit, but it's not huge.

Or more consistently foo.len() which we *almost* have except it's spelled __len__ :P I agree this is a minor nit, but also rather pointless as far as I can tell.

For the language developers, does it matter? Again, it's a small benefit. It's still our job to design it, not a third-party. Syntax and stdlib are both part of the language.

All the things you think we might avoid by being more "functional", we'd still have to do them, saving nothing. Except all the details suggest *worse* solutions would be found, as we wouldn't be fine-tuning the syntax for the use-cases.

The only genuine use-case for your changes I've seen in this whole thread is the dispatch-dict, but your changes are far too distract to justify it.

I'm not surprised. In fact, after programming in Python myself for a decade or so I'm surprised I was able to find any suitable example ;-) The Sapir-Whorf hypotheses predicts this. A Lisp or Haskell programmer would probably be much better armed than I to provide concrete examples. Even if you don't subscribe to Sapir-Whorf, I think you could probably agree that at least the lack of idioms for FP in Python would contribute to our mutual inability to visualize particularly useful examples.

As an aside, I can't speak for Guido, but I haven't seen any indication python's development process may collapse. It's a non-problem.

I didn't mean the development process may collapse, I mean the rejection process may collapse due to pressure (and I believe it has already on occasion). Regards, Cliff

On Fri, 2008-09-12 at 21:56 -0600, Adam Olsen wrote:

At most, it's a *small* benefit to the language. It comes a tradeoff, just like a+b rather than add(a, b), or len(x) rather than x.len().

Okay, you kept providing this example and I kept brushing it off because I didn't see where you were leading. Now I do and now I realize that I've utterly failed to get the problem across. Let's use your example from above: a+b vs add(a, b) a + b is the *exact* semantical equivalent of add(a, b). They do the same thing, they can be used in the same contexts. The + operator is pure semantic sugar (and well-used sugar too). Now let's imagine things aren't quite so equivalent. a+b has the same result as add(a, b), except you can only use the + operator in one particular context: assignment. It cannot be used say, with a comparison operator. So in my imaginary Python, this is legal x = 1 + 2 but this is not if x < y + 1: pass and must instead be written if x < add(y, 1): pass Now lets make it even more of an exact comparison. Let's say the add(x,y) function didn't even exist. You have to write it yourself. Worse, writing it requires fairly deep knowledge of Python, and worse yet, you sometimes must write a different add(x,y) function depending on what context you want to use it in. At this point we've now made your example much closer in spirit to the problem I'm attempting to present.

You're claiming we don't have to make that tradeoff.. but you've done *nothing* to substantiate that. Even logix only adds more complexity, rather than removing and unifying existing syntax.

And you've provided nothing to substantiate this (aside from this assertion). Also, if we combine statements into expressions, how does this not unify existing syntax? Combining two into one is practically the definition of "unify".

We don't care about purity. Statements vs expressions.. functional vs imperative.. they're tools, pure and simple. We switch back and forth as appropriate. Most of the time quietly, subtly, such that you usually don't even notice we've done it. We like it that way.

Who is this "we"? People who use Python? Am I not a person who uses Python (and for the better part of 10 years I might add). I'm not some outsider saying "can you add X from my previously favorite language Y so I can program the way I used to". I'm someone who, until maybe a year ago, was just as narrow-minded about Python's shortcomings as you are now. I mean seriously, has it occurred to you that it's possible that you simply aren't getting my point?

You'll be a lot more productive if you learn to relax, concerning yourself with use-cases, not ideals.

Thanks for the advice. Should I also read a good book, maybe consider a massage? Regards, Cliff

On Sat, 2008-09-13 at 12:27 -0700, Bruce Leban wrote:

On Sat, Sep 13, 2008 at 9:02 AM, Cliff Wells <cliff@develix.com> wrote: Let's use your example from above: a+b vs add(a, b)

a + b is the *exact* semantical equivalent of add(a, b). They do the same thing, they can be used in the same contexts. The + operator is pure semantic sugar (and well-used sugar too).

First, + is *syntactic* sugar not semantic sugar. Syntactic sugar is something that changes the appearance but not the semantics.

Sorry, clearly typing too fast. I meant semantic the first time I used it and syntactic the second.

C uses semicolons and Python doesn't.

Python does, but only as a substitute for a newline.

Is that syntactic sugar? Can I just ask C to remove the "restriction" that semicolons are required without defining precisely what that means?

Arguably you'd end up with what Python does now. I think that's well-enough defined (C's for-loop being an exception, obviously).

If I do, then I end up with something like Javascript's semicolon rule which is confusing and error-prone. No thanks.

I'm not aware of JavaScript's semicolon rule. When I write JS I use it as a line-terminator.

Now let's imagine things aren't quite so equivalent. a+b has the same result as add(a, b), except you can only use the + operator in one particular context: assignment. It cannot be used say, with a comparison operator.

So in my imaginary Python, this is legal

x = 1 + 2

but this is not

if x < y + 1: pass

and must instead be written

if x < add(y, 1): pass

This is a strawman. You hypothesize that + would only be usable in the context of "assignment". But that's not the context. The context is expression. The semantics are the same.

I was attempting to use Adam's example (which I would have labeled a strawman, except I don't believe it was intentional) in order to convey the difference between what he was saying and what I was saying. If it fails, well, it's not the example I would have chosen, but I felt it might help Adam and I find common ground for debating on, so <shrug>.

Here's a real example: consider a language where +, *, etc. can only be used in the context of augmented assignment. That is: a += b c *= d etc. while I have to use functional syntax everywhere else: x = add(a,b) y = mul(c,d) etc. Now I propose improving this language by eliminating the "restriction" that +, * etc. can only be used in assignments.

Actually, this is the strawman. If you read back (yes, I'm asking a lot) you'll note that I asserted that statements are (or could be) semantically equivalent to expressions except they have a limited context. Any statement can be made into an expression (but not the reverse). T

You ask me to define the syntax and semantics of these operators and I brush that off as unimportant. All I'm advocating is "lifting a restriction" in the language. I'm unifying syntax not adding complexity, right? No need to explain pesky details.

No. That's ridiculous. You might be surprised when you use my new features that x = 2 * 3 + 4 y = 2 - 3 + 4 sets x to 14 and y to -5 but that's not my fault. (And lest you say this is a strawman and no one would consider any value other than 10 and 3, go look at http://tinyurl.com/27ttcy.)

It's a matter of defining ambiguity. APL does it by being strictly left-to-right and Python (and most other languages) defines it by a (admittedly more complicated) algebraic order of operations.

This is what you're doing. If you can't precisely define how the language is different with your proposed change, then it's not worth discussing. You can do that with a formal definition as I suggested or by actually implementing your changes as Guido suggested.

I think providing examples is enough for a general discussion of the idea. Further, if anyone were really curious beyond just discussing it in the abstract, they could try Logix (although you'll unfortunately need to have Python 2.4 to do so). I understand that any such change would need to be adequately defined. But I don't consider the discussion to have progressed to that point. If people do not even understand that expression-oriented languages provide any advantage (in fact, many of them apparently consider it a disadvantage), then there's little point in discussing what particular syntax this apparently useless change would take on.

You can't do it by insisting that no one understands what you're talking about.

When someone suggests that the distinction between expressions and statements is syntactic sugar on the same level as the + operator, then I have no choice but to assume they really don't understand what I'm talking about. Whether that's a failing on my part or theirs is an open question, but there's no doubt about the lack of understanding. Anyway, to be frank, once I discovered that the developer of Logix had moved to Ruby, I finally admitted to myself that I was being completely unreasonable in expecting Python to be any more than what it is. I've already made up my mind to move to a language I find more suitable. Regards, Cliff

On Sat, 2008-09-13 at 19:47 -0600, Adam Olsen wrote:

On Sat, Sep 13, 2008 at 3:58 PM, Cliff Wells <cliff@develix.com> wrote:

...

On Sat, 2008-09-13 at 12:27 -0700, Bruce Leban wrote:

...

You can't do it by insisting that no one understands what you're talking about.

When someone suggests that the distinction between expressions and statements is syntactic sugar on the same level as the + operator, then I have no choice but to assume they really don't understand what I'm talking about. Whether that's a failing on my part or theirs is an open question, but there's no doubt about the lack of understanding.

That much we can agree on.

Throughout this thread very little if any progress has been made. I'm willing to continue on IRC if you'd like, but I won't clutter the list any more.

Fair enough. I'm actually willing to let it go at this point. If someone has a point to make or a question, I won't ignore them, but I'll refrain from indulging myself any further. And seriously, sincere thanks for the debate. As I mentioned early on, I didn't really expect this to go anywhere productive (except to the extent it let me get it off my chest), so I appreciate everyone's patience. Regards, Cliff