RE: [Python-Dev] exec/with thunk-handling proposal
holger krekel wrote:
Michael Hudson wrote:
holger krekel
writes: I think we can may get away with only a "weak" keyword and allow the aforementioned encapsulation of execution events into an object like this:
exec expr [with params]: suite
Gut reaction: ugh!
then i guess you rather prefer new keywords. I thought that there is a general reluctance to introduce new keywords *and* many people dislike 'exec' for its existence. So reusing it (and we are dealing with execution aspects IMO) makes some sense to me.
I can see your point. Personally, I don't have any great problems with "exec", so I'd like to understand better how your proposal integrates this new use of exec with the existing use. And at some point, it will be necessary to confirm that the parser can be made to distinguish the various uses (it seems to me that extra lookahead will be needed).
where the expression is evaluated to return a "thunk" handler with these optional "execution" hooks:
def __enter__(self): "before suite start"
def __except__(self, type, value, tb): "swallow given exception, reraise if neccessary"
def __leave__(self): """upon suite finish (not called if __except__ exists and an exception happened) """
The above "with" parameters (of the form name=expr, comma-separated) are bound in local (or global/nested) *and* handler instance namespace. The 'suite' is what we call "thunk".
The above logic allows clean timely finalization for *multiple* ressources:
exec autoclose() with f1=open(name1), f2=open(name2, 'w'): for line in f1: ... f2.write(...)
That looks messy.
as compared to? I think i have seen messier ideas with all kinds of brackets and double-colons :-)
I agree with you here. This looks OK to me in terms of syntax. I'm not so sure of the semantics - that may be the bit Michael thought was "messy". Binding the with parameters in two places feels odd, and in particular injecting the parameter names into the instance namespace is unusual (I know of no other construct in Python which implicitly changes an instance namespace like this). And as far as the local namespace is concerned, consider exec something() with f1=12: suite # Out of the exec now - what is the value of f1??? I assume that the assignment to f1 is still in existence, otherwise you're inventing a new way of defining a scope. But if the binding to f1 does still exist, then that looks odd, because "exec ... with f1=12" *reads* as if f1 is only in scope for the duration of the statement. You seem to lose either way...
which would execute as follows
a) autoclose() instance is created and stored as the "thunk"-handler
We need a name for these. I've been using "monitor" for a while, but I'm not sure it's that apposite.
So far i'd liked "execution handler" because 'enter/exit/except' are IMHO execution events which you can hook into. Do you agree with calling them 'execution events' or how would you call them?
The way you've defined things, I see the logic - it's hooks into the execution model. I'm not sure I'd have thought of that way of looking at it for myself, though...
b) f1/f2 are stored as attributes on the autoclose instance
c) f1/f2 are put into the local/global namespace (and nested ones if rebinding is allowed)
d) thunk executes (for line ...)
e) autoclose 'leave' hook is called (with or without exception) and is implemented like this:
def __leave__(self): for obj in self.__dict__.values(): obj.close()
But what if the object was more complex, and needed its own state? How could it tell which were "its own" variables and which had been injected? (Other than by hard coding a list of "its own" variables to ignore, which is utterly horrible).
f) thunk handler is removed
"Too much magic!"
Hmmm. That seems a bit unfair to me as many other proposals didn't care to elaborate the exact sequence. You probably are refering to the "namespace interactions" as the other stuff is probably the same with any other proposal so far.
No, I agree with Michael here. Just because other proposals handwave and don't give proper semantics doesn't mean that they don't also have too much magic. In my book "unclear semantics" is a worse accusation than "too much magic". But both are bad :-)
Because computing 'f1' may succeed but 'f2' can subsequently fail the assignments *have to* execute within "autoclose" control.
Hmm. In the equivalent try...finally construct, the idiom is to open the file *outside* the try statement, so that exceptions when opening the file don't get accidentally caught. You need to address this - particularly in the case of multiple variables.
Would the following be the correct way to achieve this with your patch?
f1=open(inputfn) with autoclose(f1): f2 = open(outputfn, 'w') with autoclose(f2): for line in f1: ... f2.write(line)
I think there should be a better solution for multiple ressources.
In many ways, I agree with you. I originally wanted multiple expressions, and assignments within the "with" statement. But people argued against them (look back over the thread if you want the details). I honestly don't believe that the with statement will ever please everyone - the best we can hope for is to let it handle the simple cases well, with clean, well-defined semantics with as few surprises as possible, and leave the remaining cases to the existing try statement (or to a more ambitious thunk-based mechanism). There is a real risk of slimming the with statement down to uselessness by this philosophy. But this may simply indicate that the need is perceived rather than real :-)
Now on to the usage of the except hook. Nice use cases might be
exec retry(maxretry=3, on=IOError): # do network io-stuff or exec skip_on(AttributeError, TypeError): some_object.notify_hook()
but i am sure there are more. Exception handling is often ugly when inlined with the code. I think that stating 'exception behaviour' up-front allows to write nice readable constructs.
I am *still* not convinced that an __except__ hook is worth the pain. Can you implement those for me?
Maybe, i haven't looked at your implementation yet and there are some other projects pending :-)
There's no implementation of any form of except hook in existence yet (excluding the IEXEC patch). The key point here is that your explanation of semantics above didn't cover __except__ hooks. I'd like to see __except__ integrated into the semantics you describe, and then with that as a basis I's like to see definitions of retry() and skip_on(). The idea looks interesting, but I'd like to see concrete code.
yields already can't go in blocks with finally statements, right?
right.
Would you propose calling the __enter__ method each time the generator resumed?
yes, neccessary for locking (as the above example tried to indicate).
But not for files. It's arguable that you need __enter__, __exit__, __yield__, and __resume__ hooks for full generality. On the other hand, YAGNI makes more sense to me...
If there is interest i can probably modify my patch to allow all of the proposed syntax so that you could play around with it.
Working code for *any* of these ideas could only be a good thing. User code which benefits from the new features is also good. The auto-closing file example has been worked to death, and its main effect for me so far has been to make me wonder whether we need *any* of these mechanisms :-) The lock example is better, but still pretty borderline. I'd really, really like to see a substantial, non-toy use case. Actually, my main experience is C/C++ based, where blocks with many memory allocations and the occasional open file is the canonical example. Freeing and closing in reverse order, under all error conditions, is a *pain*. But for this, the C++ "resource acquisition is initialisation" idiom works brilliantly. It's a shame it can't be made to work in Python. Paul.
On Tue, Feb 04, 2003 at 01:07:37PM -0000, Moore, Paul wrote:
And as far as the local namespace is concerned, consider
exec something() with f1=12: suite # Out of the exec now - what is the value of f1???
I assume that the assignment to f1 is still in existence, otherwise you're inventing a new way of defining a scope. But if the binding to f1 does still exist, then that looks odd, because "exec ... with f1=12" *reads* as if f1 is only in scope for the duration of the statement. You seem to lose either way...
But Python already has this "problem": for i in range(5): pass print i (What's more, 'i' will have a different value than 'i' in the "equivalent" C fragment int i; for(i=0; i<5; i++) /*NOTHING*/; printf("%d\n", i); ) But this doesn't seem to bother anybody. Does it? Jeff
participants (2)
-
Jeff Epler
-
Moore, Paul