Re: [Python-Dev] Proposed resolutions for open PEP 343 issues
At 09:19 AM 10/23/2005 -0700, Guido van Rossum wrote:
On 10/23/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
However, I'm still concerned about the fact that the following class has a context manager that doesn't actually work:
class Broken(object): def __context__(self): print "This never gets executed" yield print "Neither does this"
That's only because of your proposal to endow generators with a default __context__ manager. Drop that idea and you're golden.
(As long as nobody snuck the proposal back in to let the with-statement silently ignore objects that don't have a __context__ method -- that was rejected long ago on.)
Actually, you've just pointed out a new complication introduced by having __context__. The return value of __context__ is supposed to have an __enter__ and an __exit__. Is it a type error if it doesn't? How do we handle that, exactly? That is, assuming generators don't have enter/exit/context methods, then the above code is broken because its __context__ returns an object without enter/exit, sort of like an __iter__ that returns something without a 'next()'.
On 10/23/05, Phillip J. Eby <pje@telecommunity.com> wrote:
Actually, you've just pointed out a new complication introduced by having __context__. The return value of __context__ is supposed to have an __enter__ and an __exit__. Is it a type error if it doesn't? How do we handle that, exactly?
That is, assuming generators don't have enter/exit/context methods, then the above code is broken because its __context__ returns an object without enter/exit, sort of like an __iter__ that returns something without a 'next()'.
I would have thought that the parallel with __iter__ would be the right way to go:
class C: ... def __iter__(self): ... return 12 ... c = C() iter(c) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: __iter__ returned non-iterator of type 'int'
So, when you try calling __context__ in a with statement (or I guess in a context() builtin if one were to be added), raise a TypeError if the resulting object doesn't have __enter__ and __exit__ methods. (Or maybe just if it has neither - I can't recall if the methods are optional, but certainly having neither is wrong). Paul.
On 10/23/05, Phillip J. Eby <pje@telecommunity.com> wrote:
Actually, you've just pointed out a new complication introduced by having __context__. The return value of __context__ is supposed to have an __enter__ and an __exit__. Is it a type error if it doesn't? How do we handle that, exactly?
Of course it's an error! The translation in the PEP should make that quite clear (there's no testing for whether __context__, __enter__ and/or __exit__ exist before they are called). It would be an AttributeError.
That is, assuming generators don't have enter/exit/context methods, then the above code is broken because its __context__ returns an object without enter/exit, sort of like an __iter__ that returns something without a 'next()'.
Right. That was my point. Nick's worried about undecorated __context__ because he wants to endow generators with a different default __context__. I say no to both proposals and the worries cancel each other out. EIBTI. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Right. That was my point. Nick's worried about undecorated __context__ because he wants to endow generators with a different default __context__. I say no to both proposals and the worries cancel each other out. EIBTI.
Works for me. That makes the resolutions for the posted issues: 1. The slot name "__context__" will be used instead of "__with__" 2. The builtin name "context" is currently offlimits due to its ambiguity 3a. generator-iterators do NOT have a native context 3b. Use "contextmanager" as a builtin decorator to get generator-contexts 4. The __context__ slot will NOT be special cased I'll add those into the PEP and reference this thread after Martin is done with the SVN migration. However, those resolutions bring up the following issues: 5 a. What exception is raised when EXPR does not have a __context__ method? b. What about when the returned object is missing __enter__ or __exit__? I suggest raising TypeError in both cases, for symmetry with for loops. The slot check is made in C code, so I don't see any difficulty in raising TypeError instead of AttributeError if the relevant slots aren't filled. 6 a. Should a generic "closing" context manager be provided? b. If yes, should it be a builtin or in a "contexttools" module? I'm not too worried about this one for the moment, and it could easily be left out of the PEP itself. Of the sample managers, it seems the most universally useful, though. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com
On 10/24/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
That makes the resolutions for the posted issues:
1. The slot name "__context__" will be used instead of "__with__" 2. The builtin name "context" is currently offlimits due to its ambiguity 3a. generator-iterators do NOT have a native context 3b. Use "contextmanager" as a builtin decorator to get generator-contexts 4. The __context__ slot will NOT be special cased
+1
I'll add those into the PEP and reference this thread after Martin is done with the SVN migration.
However, those resolutions bring up the following issues:
5 a. What exception is raised when EXPR does not have a __context__ method? b. What about when the returned object is missing __enter__ or __exit__? I suggest raising TypeError in both cases, for symmetry with for loops. The slot check is made in C code, so I don't see any difficulty in raising TypeError instead of AttributeError if the relevant slots aren't filled.
Why are you so keen on TypeError? I find AttributeError totally appropriate. I don't see symmetry with for-loops as a valuable property here. AttributeError and TypeError are often interchangeable anyway.
6 a. Should a generic "closing" context manager be provided?
No. Let's provide the minimal mechanisms FIRST.
b. If yes, should it be a builtin or in a "contexttools" module? I'm not too worried about this one for the moment, and it could easily be left out of the PEP itself. Of the sample managers, it seems the most universally useful, though.
Let's leave some examples just be examples. I think I'm leaning towards adding __context__ to locks (all types defined in tread or threading, including condition variables), files, and decimal.Context, and leave it at that. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Almost there - this is the only issue I have left on my list :) Guido van Rossum wrote:
On 10/24/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
However, those resolutions bring up the following issues:
5 a. What exception is raised when EXPR does not have a __context__ method? b. What about when the returned object is missing __enter__ or __exit__? I suggest raising TypeError in both cases, for symmetry with for loops. The slot check is made in C code, so I don't see any difficulty in raising TypeError instead of AttributeError if the relevant slots aren't filled.
Why are you so keen on TypeError? I find AttributeError totally appropriate. I don't see symmetry with for-loops as a valuable property here. AttributeError and TypeError are often interchangeable anyway.
The reason I'm keen on TypeError is because 'abstract.c' uses it consistently when it fails to find a method to support a requested protocol. None of the abstract object methods currently raise AttributeError, and this property is fairly visible at the Python level because the abstract API's are used to implement many of the bytecodes and various builtin functions. Both for loops and the iter function, for example, get their current exception behaviour from PyObject_GetIter and PyIter_Next. Having had a look at mwh's patch, however, I've realised that going that way would only be possible if there were dedicated bytecodes for GET_CONTEXT, ENTER_CONTEXT and EXIT_CONTEXT (similar to the dedicated GET_ITER and FOR_ITER). Leaving the exception as AttributeError means that level of bytecode hacking isn't necessary (mwh's patch just emits a fairly normal try/finally statement, although it still modifies the bytecode to include LOAD_EXIT_ARGS). So, the inconsistency with other syntactic protocols still bothers me, but I can live with AttributeError if you don't want to add three new bytecodes just to support PEP 343. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com
On 10/25/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
Almost there - this is the only issue I have left on my list :) [,,,]
Why are you so keen on TypeError? I find AttributeError totally appropriate. I don't see symmetry with for-loops as a valuable property here. AttributeError and TypeError are often interchangeable anyway.
The reason I'm keen on TypeError is because 'abstract.c' uses it consistently when it fails to find a method to support a requested protocol.
Hm. abstract.c well predates the new type system. Slots and methods weren't really unified back then, so TypeError made obvious sense at the time. But with the new unified types/classes, those TypeErrors are really just delayed (or precomputed? depends on your POV) AttributeErrors.
None of the abstract object methods currently raise AttributeError, and this property is fairly visible at the Python level because the abstract API's are used to implement many of the bytecodes and various builtin functions. Both for loops and the iter function, for example, get their current exception behaviour from PyObject_GetIter and PyIter_Next.
Having had a look at mwh's patch, however, I've realised that going that way would only be possible if there were dedicated bytecodes for GET_CONTEXT, ENTER_CONTEXT and EXIT_CONTEXT (similar to the dedicated GET_ITER and FOR_ITER).
Leaving the exception as AttributeError means that level of bytecode hacking isn't necessary (mwh's patch just emits a fairly normal try/finally statement, although it still modifies the bytecode to include LOAD_EXIT_ARGS).
Let's definitely not introduce new bytecodes just so we can raise a different exception.
So, the inconsistency with other syntactic protocols still bothers me, but I can live with AttributeError if you don't want to add three new bytecodes just to support PEP 343.
I think the consistency you are seeking is a mirage. The TypeErrors stem from the pre-computation of the slot population, not from some requirements to raise TypeError for failing to implement some required built-in protocol. I wouldn't hold it against other implementations of Python if they raised AttributeError in more situations. It is true though that AttributeError is somewhat special. There are lots of places (perhaps too many?) where an operation is defined using something like "if the object has attribute __foo__, use it, otherwise use some other approach". Some operations explicitly check for AttributeError in their attribute check, and let a different exception bubble up the stack. Presumably this is done so that a bug in somebody's __getattr__ implementation doesn't get masked by the "otherwise use some other approach" branch. But this is relatively rare; most calls to PyObject_GetAttr just clear the error if they have a different approach available. In any case, I don't see any of this as supporting the position that TypeError is somehow more appropriate. An AttributeError complaining about a missing __enter__, __exit__ or __context__ method sounds just fine. (Oh, and please don't go checking for the existence of __exit__ before calling __enter__. That kind of bug is found with even the most cursory testing.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
It is true though that AttributeError is somewhat special. There are lots of places (perhaps too many?) where an operation is defined using something like "if the object has attribute __foo__, use it, otherwise use some other approach". Some operations explicitly check for AttributeError in their attribute check, and let a different exception bubble up the stack. Presumably this is done so that a bug in somebody's __getattr__ implementation doesn't get masked by the "otherwise use some other approach" branch. But this is relatively rare; most calls to PyObject_GetAttr just clear the error if they have a different approach available. In any case, I don't see any of this as supporting the position that TypeError is somehow more appropriate. An AttributeError complaining about a missing __enter__, __exit__ or __context__ method sounds just fine. (Oh, and please don't go checking for the existence of __exit__ before calling __enter__. That kind of bug is found with even the most cursory testing.)
Hmmm... Would it be reasonable to introduce a ProtocolError exception? --eric
On 10/25/05, Eric Nieuwland <eric.nieuwland@xs4all.nl> wrote:
Hmmm... Would it be reasonable to introduce a ProtocolError exception?
And which perceived problem would that solve? The problem of Nick & Guido disagreeing in public? -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
On 10/25/05, Eric Nieuwland <eric.nieuwland@xs4all.nl> wrote:
Hmmm... Would it be reasonable to introduce a ProtocolError exception?
And which perceived problem would that solve? The problem of Nick & Guido disagreeing in public?
;-) No, that will go on in other fields, I guess. It was meant to be a bit more informative about what is wrong. ProtocolError: lacks __enter__ or __exit__ --eric
[Eric "are all your pets called Eric?" Nieuwland]
Hmmm... Would it be reasonable to introduce a ProtocolError exception?
[Guido]
And which perceived problem would that solve?
[Eric]
It was meant to be a bit more informative about what is wrong.
ProtocolError: lacks __enter__ or __exit__
That's exactly what I'm trying to avoid. :) I find "AttributeError: __exit__" just as informative. In either case, if you know what __exit__ means, you'll know what you did wrong. And if you don't know what it means, you'll have to look it up anyway. And searching for ProtocolError doesn't do you any good -- you'll have to learn about what __exit__ is and where it is required. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
[Eric "are all your pets called Eric?" Nieuwland]
Hmmm... Would it be reasonable to introduce a ProtocolError exception?
[Guido]
And which perceived problem would that solve?
[Eric]
It was meant to be a bit more informative about what is wrong.
ProtocolError: lacks __enter__ or __exit__
That's exactly what I'm trying to avoid. :)
I find "AttributeError: __exit__" just as informative. In either case, if you know what __exit__ means, you'll know what you did wrong. And if you don't know what it means, you'll have to look it up anyway. And searching for ProtocolError doesn't do you any good -- you'll have to learn about what __exit__ is and where it is required.
I see. Then why don't we unify *Error into Error? Just read the message and know what it means. And we could then drop the burden of exception classes and only use the message. A sense of deja-vu comes over me somehow ;-)
Guido van Rossum wrote:
On 10/25/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
Almost there - this is the only issue I have left on my list :) [,,,]
Why are you so keen on TypeError? I find AttributeError totally appropriate. I don't see symmetry with for-loops as a valuable property here. AttributeError and TypeError are often interchangeable anyway. The reason I'm keen on TypeError is because 'abstract.c' uses it consistently when it fails to find a method to support a requested protocol.
Hm. abstract.c well predates the new type system. Slots and methods weren't really unified back then, so TypeError made obvious sense at the time.
Ah, I hadn't considered that, because I never made significant use of any Python versions before 2.2. Maybe there's a design principle in there somewhere: Failed duck-typing -> AttributeError (or TypeError for complex checks) Failed instance or subtype check -> TypeError Most of the functions in abstract.c handle complex protocols, so a simple attribute error wouldn't convey the necessary meaning. The context protocol, on the other hand, is fairly simple, and an AttributeError tells you everything you really need to know. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com
On 10/25/05, Nick Coghlan <ncoghlan@gmail.com> wrote:
Maybe there's a design principle in there somewhere:
Failed duck-typing -> AttributeError (or TypeError for complex checks) Failed instance or subtype check -> TypeError
Doesn't convince me. If there are principles at work here (and not just coincidences), they are (a) don't lightly replace an exception by another, and (b) don't raise AttributeError; the getattr operation raise it for you. (a) says that we should let the AttributeError bubble up in the case of the with-statement; (b) explains why you see TypeError when a slot isn't filled.
Most of the functions in abstract.c handle complex protocols, so a simple attribute error wouldn't convey the necessary meaning. The context protocol, on the other hand, is fairly simple, and an AttributeError tells you everything you really need to know.
That's what I've been saying all the time. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (5)
-
Eric Nieuwland -
Guido van Rossum -
Nick Coghlan -
Paul Moore -
Phillip J. Eby