How about doing it both ways: if you try to pass start/stop for an iterator instead of a sequence, you get an AttributeError on __getindex__. (I'm not proposing this, just throwing it out as an idea. It does make explaining it more difficult, which is an argument against.)

The more complexity is suggested, the more I like my simplest variant. --Guido van Rossum (home page: http://www.python.org/~guido/)

...

1. indexed()

...

I don't like the name "indexed"; adjectives do not make good function names. Maybe iterindexed()?

Any reasonable name is fine with me. Originally, I proposed index() but was talked out of it.

I like itercount() or enumerate(). The reason is that this function can work with any iterable including those that do not have numeric indices (such as dictionaries). Counting or enumeration is what is really happening.

I don't like either of those, you don't seem to like iterindexed(), so we'll have to think more about a name.

for linenum, line in itercount( alinelist, 1 ): print 'Line %03d: %s' % (linenum, line)

You can do that like this: for linenum, line in itercount(alinelist): print 'Line %03d: %s' % (linenum+1, line)

In summary, here is my order of preference: 1. itercount(collection, countfrom=0, stop=None) # best 2. itercount(collection, countfrom=0) # almost excellent 3. itercount(collection) # good enough

I really hate the alternate count option, so let's agree to pick 3 (with a different name).

...

2. Generator comprehensions

If implementation effort were taken out of the equation, would the balance shift back to a +1 ? In other words, if I find a developer who wants this badly enough to work with me to get it implemented, do we have a go?

Not really. After it's written (and debugged!), it's still a lot of code that needs to be maintained (we're already thinking about rewriting the bytecode generators, there's Jython, and so on) for a very slight benefit. What other feature would you give up in order to have this on? If the only way to get you to stop asking for this is a -1 from me, I'll give it a -1.

Several commenters wanted this one quite a bit. An excerpt: "This rules. You rock."

Yeah, if I left Python's design to Ping, it would become quite the clever hack. :-)

...

3. Generator exception passing

I need help from others on py-dev who can articulate the need clearly. For me, it's as plain as day and I don't know what to say to convey the message better than it is expressed in the PEP.

Too bad. This one gets a big fat -1 until there's a good motivational section. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Fri, 29 Mar 2002, Just van Rossum wrote:

But the *reason* for this proposal is to turn this idiom:

for i in range(len(seq)): element = seq[i] ...

into this:

for i, element in enumerate(seq): ...

It's hardly useful for types that don't have numeric indices, so the fact that it _does_ work with any iterator seems almost an implementation detail.

I quite like the name enumerate. Hate itercount. I'm neutral on indexed.

Pardon me if I'm daft, but... What was wrong with seq.items() / seq.iteritems(), that they dropped out of the discussion ? /Paul

[Raymond Hettinger, on generator exceptions]

I need help from others on py-dev who can articulate the need clearly. For me, it's as plain as day and I don't know what to say to convey the message better than it is expressed in the PEP.

I can't: "simple generators" were deliberately not coroutines, and I dislike trying to add 2-way control-flow gimmicks to them. Over the years I've found that "resumable function" is easy to get across even to C programmers <wink>, but anything fancier hits a wall. So I'd like to keep generators simple. Generator exceptions do less damage (IMO) to simplicity than some of the other extensions, but in practice, in the very few cases I've wanted something like that, this worked fine: class Whatever: def __init__(self): self._stop = 0 def stop(self): self._stop = 1 def generator(self): while 1: ... yield something if self._stop: break cleanup I agree a throw() would be easier to live with, the problem is that I've so rarely needed it.

... It was meant to be a generic and oversimplified example of clean-up code in a generator being used as a data consumer.

Simple generators were designed to be data producers, and:

... Dr. Mertz's co-routine code does make a much better case,

Bingo. If you want coroutines, design a coroutine facility. Piling more semantics on to "yield" takes a delightfully simple yet powerful gimmick and hides it under ten kinds of obscurity.

[Tim]

So I'd like to keep generators simple.

I hear you.

Generator exceptions do less damage (IMO) to simplicity than some of the other extensions, but in practice, in the very few cases I've wanted something like that, this worked fine:

class Whatever: def __init__(self): self._stop = 0

def stop(self): self._stop = 1

def generator(self): while 1: ... yield something if self._stop: break cleanup

Hmm, the client code needs to maintain TWO instance variables, the generator object and the instance of Whatever. It works but doesn't smell right to me: w = Whatever() wg = w.generator() wg.next() wg.next() w.stop(); wg.next() # trigger clean-up

Bingo. If you want coroutines, design a coroutine facility. Piling more semantics on to "yield" takes a delightfully simple yet powerful gimmick and hides it under ten kinds of obscurity.

I don't see the obscurity. This is the kind of tool that is invisible and out of the way until the one day you really need it and the tool is right there in your hand (as if by time machine). Throw wouldn't be used 98% of the time. So, simple things are left simple and hard things are a little less hard. I didn't come up with these ideas out of the ether. It came up in a real world application for producing index prints from jpegs. That being said, I accept that generator parameter passing and exception passing are doomed. Here is my alternate proposal that is much simpler and doesn't encourage weirdness. [Developers feel free to stomp on the idea but please don't smack me around for thinking it up]. Specification for Generator Attributes: 1. Allow attributes to be assigned to generator objects. 2. Provide a means of accessing those attributes from within the generator by using a reference to __self__. def mygen(): print __self__.greeting yield 'hello' print __self__.greeting g = mygen() g.greeting = 'Good morning' print g.next() g.greeting = 'Good night' Advantages: Uses the standard python attribute assignment idiom so there is no learning curve and no surprises. Easily implementable without new keywords, new builtins, or parser magic. Provides a data sharing solution that avoids the yield / next() matching problem, avoids enclosing classes, and avoid global variables. Simple and neat. Disads: Introduces a new system variable, __self__. Raymond Hettinger, CPA P.S. No one needs to say anything mean to make me go away. This is the last generator proposal and then I'm back off to volunteering on Py-Help and co-authoring a Python book update.

On Sat, Mar 30, 2002, jepler@unpythonic.dhs.org wrote:

I had a bit of confusion that there's not StopIteration traceback printed, and neither is "after that". Apparently an uncaught StopIteration call just exits silently? (I'm using 2.3a0 here, in the case that it matters---cvs from 2-3 weeks ago)

Yeah, just like IndexError in a for loop. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ Why is this newsgroup different from all other newsgroups?

[Tim]

...

Bingo. If you want coroutines, design a coroutine facility. Piling more semantics on to "yield" takes a delightfully simple yet powerful gimmick and hides it under ten kinds of obscurity.

[Raymond Hettinger]

I don't see the obscurity.

I know <wink>. As I said earlier in the msg, "generator exceptions do less damage (IMO) to simplicity than some of the other extensions". I had in mind above *all* the other gimmicks suggested to be layered on top of yield, including burying yield in what otherwise looked like listcomps, and the "generator parameter passing" idea.

This is the kind of tool that is invisible and out of the way until the one day you really need it and the tool is right there in your hand (as if by time machine).

Throw wouldn't be used 98% of the time.

I'd call it 99+%, but that was my original point: it's rarely needed. The (very) few times I've needed it, I was happy rolling my own.

... I didn't come up with these ideas out of the ether. It came up in a real world application for producing index prints from jpegs.

The question is then whether simple generators are a fork that hasn't yet grown enough tines, or whether your app really wanted a knife. That is, if you try to use a feature and find it's not enough for a specific problem, it doesn't follow that the feature is flawed. BTW, the cleanest way I can think of to implement generator exceptions would be to add a new opcode to the Python virtual machine, and that's a pretty heavy change for a rarely-needed minor convenience.

That being said, I accept that generator parameter passing and exception passing are doomed.

Heh. I started pushing for generators in the very early 90's. The idea was rejected multiple times, but usually because somebody else hijacked the idea in favor of continuations instead. Pare away the over-elaborate, and something simple that remains may have a chance, if you just keep poking at it for a decade <wink>.

Here is my alternate proposal that is much simpler and doesn't encourage weirdness. [Developers feel free to stomp on the idea but please don't smack me around for thinking it up].

Specification for Generator Attributes:

1. Allow attributes to be assigned to generator objects.

That's doable. You can already assign attributes to generator-functions, so I figure this means generator-iterators (to which you cannot currently assign attributes).

2. Provide a means of accessing those attributes from within the generator by using a reference to __self__.

<

def mygen(): print __self__.greeting yield 'hello' print __self__.greeting

g = mygen() g.greeting = 'Good morning' print g.next() g.greeting = 'Good night'

This isn't an, umm, compelling example. Why would I *really* want to do this? If I really wanted to, I expect I'd again make the generator a method of a class and use vanilla instance variables -- or maybe I would really want coroutines.

Advantages: Uses the standard python attribute assignment idiom so there is no learning curve and no surprises.

Ditto for instance variables.

Easily implementable without new keywords, new builtins, or parser magic.

That isn't so: the parser is going to have to recognize "__self__" as a special token, and generate special code to make it resolve to the generator-iterator object derived from the generator-function. This isn't at all like the current use of "self" in methods, because in that context "self" is just a named parameter like any other, and lives in the function's local namespace like any other named parameter. This new gimmick would make more sense if the generator's 'def' were spelled def mygen(__self__): but then it gets uglier if mygen is also a method.

Provides a data sharing solution that avoids the yield / next() matching problem, avoids enclosing classes, and avoid global variables. Simple and neat.

I don't think it's so simple, as there's nothing else like it in Python now: it can't really be explained in terms of anything that already exists. Names always live in the local, global or builtin namespaces. __self__ would have to live in the local namespace to make any sense, but *no* name shows up "by magic" in a local namespace now.

Disads: Introduces a new system variable, __self__.

The rub is that it's a system variable unlike any other. You may find it's more convenient to pass an explicit "message carrier" to mygen: class Carrier: pass # just a dummy to hold names def mygen(__self__): __self__.this and __self__.that, exactly as above msgs = Carrier() g = mygen(msgs) msgs.greeting = 'Good morning' print g.next() etc. If this is actually a common pattern for you, a very simple wrapper class could capture it, so that you could write g = Wrapper(mygen) g.greeting = 'Good morning') etc.

P.S. No one needs to say anything mean to make me go away.

Only Guido wants you to go away <wink>, and he's never mean, just Dutch. I don't know that generators will never be enhanced, but they're a brand-new feature to the vast bulk of Python programmers, and it would be prudent to wait a few years to see which issues come up repeatedly and don't find a reasonably simple solution via combining them with other Python features (like classes indeed: "doesn't need a class" isn't really a selling point). It took 'em a decade to get in, and they're already more powerful than in the languages Python borrowed them from (primarily Icon and CLU, with a dash of Sather).

Aahz writes:

Well, no. First-level names, yes, but each object itself is a namespace, so every name will be bound to an object that can contain names. To use my favorite example:

def f(): pass

f.permissions = 'author'

Except that those are not names, they are attributes. Namespaces nest, in Python; permissions does not live in namespace, which becomes obvious if you try to nest it def f(): return permissions f.permissions = 'author' print f() Traceback (most recent call last): File "a.py", line 5, in ? print f() File "a.py", line 2, in f return permissions NameError: global name 'permissions' is not defined Regards, Martin

...

...

...

...

"A" == aahz writes:

A> You're only correct if you define namespaces as being the set of A> nested first-level lookups. If you define "attribute" as "name A> bound to an object namespace", then we're pretty much in A> agreement. We certainly talk often enough of module namespaces; A> I think it makes sense to expand that usage to all objects. I think it's fairly common for people to talk about an object's attributes as its namespace. The language reference does not call this a namespace. In fact, the lang ref doesn't define namespace and often uses other words to discuss naming: names, blocks, scopes, environment. Perhaps namespace is too fuzzy and we should introduce a precise term to talk about the attribute "namespace" of an object. The lang ref does say that the namespace of a class definition becomes the attribute dictionary of the class. So "attribute dictionary" seems to be preferred to "object namespace" <0.5 wink>. A> It seems pretty clear to me that f.permissions does not live in A> the local/global/builtins namespace, but it sure lives in f's A> namespace. Because attributes are names like any other name and A> can be bound to any Python object, I think it lends crucial A> orthogonality to say that attributes live in object namespace. Some people will be confused that a function f has two namespaces -- the namespace for its code object and the namespace for its attributes. This reinforces my sense that we shouldn't use the term namespace when we are trying to be precise. (Or that we need a precise definition of namespace and a clear way to disambiguate the two kinds of namespaces assocaited with a function.) I'd also like to observe that the language reference doesn't say anything about how attributes of objects are resolved. The tutorial says a bit in the chapter on classes, but none of this made it to the language reference. It would be nice to flesh out these issues in the reference manual. Jeremy

Aahz writes:

I'm fine with "local scope" and "object attributes" to disambiguate them; I just think it's important that people understand that a name is a name is a name, and all names live in *some* namespace.

That isn't really true: a computed attribute lives in no namespace, instead, some function is invoked to determine the attribute value. Furthermore, some attributes live in multiple namespaces. Given obj.name what namespace is considered to find the name? NOT the namespace of obj, alone - Python also considers the namespace of obj's class (if obj is an instance), of the base classes, etc. OTOH, obj.name = None modifies the namespace of obj (unless name is a computed attribute). Regards, Martin

I don't think that setting or getting an attribute is a name binding operation. Jeremy

...

...

...

...

"JH" == Jeremy Hylton writes:

JH> I don't think that setting or getting an attribute is a name JH> binding operation. I'm curious why setting an attribute isn't a name binding operation? Are you binding the object to the attribute name in the object's attribute namespace? Since that maps to a setting in an __dict__ (usually), what /would/ you call it?

...

...

...

...

"A" == Aahz writes:

A> Maybe yes and maybe no -- we'll argue about that later. But do A> you agree that it's still a binding operation of some sort? Certainly a setter is, although we could argue about the getter (ignoring getters with side-effects). A> If so, what's the generic term for all possible things that can A> be bound *from*? (They bind to objects, of course.) I'm sure I don't follow. :) I'd say that a binding operation is that which binds an object to a name in a particular namespace. But I'd want to prefix the term "namespace" with an adjective to be precise. An object's attribute namespace, a module's namespace, etc. -Barry

...

...

...

...

"A" == Aahz writes:

A> * primary names (gotten from the lang ref), which are names in A> the local/global/builtin namespace A> * object names (and a module is an object ;-) AKA attributes I'd just call these "attributes" <wink>. Okay, maybe "attribute names". A> * computed bindings (getitem/setitem) I'd call these "computed attributes". I don't like "computed bindings" because that sounds like you're saying that you're computing both the target of the binding and the object that is being bound (or one or the other?). A> What do we call all three of these collectively? I'd say that A> I agree with MvL that computed bindings are not in fact names, "Computed attributes" aren't bindings, but that only matters if you can tell the difference. In a normal situation, if you see "obj.foo" it may not matter that "foo" only exists by virtue of a setattr/getattr/descriptor. I'd still claim that "foo" is an attribute name because it's part of the public interface for obj, albeit perhaps a virtual attribute name. A> and as I said in my other post, using "binding" as the A> collective noun looks ugly when we're also using it as a verb. A> The key is to come up with some good way of saying A> When rebinding a <foo> ... Names get bound, so names get rebound. At least, that's how I think about it. semantical-ly y'rs, -Barry

Aahz writes:

Rebinding a <foo> does not affect the originally bound object (unless the originally bound object's reference count goes to zero).

Any ideas about what to call <foo>? (Calling it a binding sounds a little too self-referential.)

That is a "name".

...

Furthermore, some attributes live in multiple namespaces. Given

obj.name

what namespace is considered to find the name? NOT the namespace of obj, alone - Python also considers the namespace of obj's class (if obj is an instance), of the base classes, etc. OTOH,

obj.name = None

modifies the namespace of obj (unless name is a computed attribute).

I'm saying that name lookup considers multiple namespaces in some cases:

...

...

class X: ... name = 1 ... x.name 1 x.__dict__.has_key("name") 0

So saying that x.name yields the value from searching the name in the namespace of x is wrong. Regards, Martin

barry wrote:

I'm curious why setting an attribute isn't a name binding operation? Are you binding the object to the attribute name in the object's attribute namespace? Since that maps to a setting in an __dict__ (usually), what /would/ you call it?

syntactic sugar? A = 1 binds 1 to the name "A" A.B = 1 calls A.__setattr__("B", 1) A["B"] = 1 calls A.__setitem__("B", 1) it's pretty clear that the first form differs from the others, but what's the difference between the second and the third form? or are all three name binding operations? </F>

On Sat, Apr 06, 2002, Fredrik Lundh wrote:

barry wrote:

...

I'm curious why setting an attribute isn't a name binding operation? Are you binding the object to the attribute name in the object's attribute namespace? Since that maps to a setting in an __dict__ (usually), what /would/ you call it?

syntactic sugar?

A = 1 binds 1 to the name "A"

A.B = 1 calls A.__setattr__("B", 1)

A["B"] = 1 calls A.__setitem__("B", 1)

it's pretty clear that the first form differs from the others, but what's the difference between the second and the third form? or are all three name binding operations?

The way I'm currently thinking of this (thanks to Barry) is that all three are target binding operations. I'm currently leaning toward attributes being names because (like primaries) they go through namespace lookup on reads, but I'm open to counter-arguments. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are times when effort is important and necessary, but this should not be taken as any kind of moral imperative." --jdecker

...

A = 1 binds 1 to the name "A"

A.B = 1 calls A.__setattr__("B", 1)

A["B"] = 1 calls A.__setitem__("B", 1)

it's pretty clear that the first form differs from the others, but what's the difference between the second and the third form? or are all three name binding operations?

I don't know if it matters for your explanation, but the *big* difference between A.B and A["B"] is that in the latter, A *is* the namespace, while in the former, A *has* a namespace (usually, but not always called A.__dict__). These namespaces are distinct, otherwise you'd have trouble keeping track of the difference between d["clear"] and d.clear... --Guido van Rossum (home page: http://www.python.org/~guido/)

barry@zope.com (Barry A. Warsaw):

I'd say that a binding operation is that which binds an object to a name in a particular namespace. But I'd want to prefix the term "namespace" with an adjective to be precise. An object's attribute namespace, a module's namespace, etc.

What about elements of a list? Calling them a "list namespace" would be rather confusing... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+