[Python-Dev] exec/with thunk-handling proposal

[holger krekel]

Michael Hudson wrote:
> holger krekel <pyth@devel.trillke.net> writes:
> > I think we can may get away with only a "weak" keyword
> > and allow the aforementioned encapsulation of execution 
> > events into an object like this:
> >
> >     exec expr [with params]: suite
> 
> Gut reaction: ugh!

then i guess you rather prefer new keywords.  I thought that there
is a general reluctance to introduce new keywords *and* many
people dislike 'exec' for its existence.  So reusing it
(and we are dealing with execution aspects IMO) makes
some sense to me.

> > where the expression is evaluated to return a
> > "thunk" handler with these optional "execution" hooks:
> >
> >     def __enter__(self):  
> >         "before suite start"
> >
> >     def __except__(self, type, value, tb): 
> >         "swallow given exception, reraise if neccessary"
> >
> >     def __leave__(self):
> >         """upon suite finish (not called if __except__ 
> >            exists and an exception happened)
> >         """
> >
> > The above "with" parameters (of the form name=expr, comma-separated) 
> > are bound in local (or global/nested) *and* handler instance 
> > namespace.  The 'suite' is what we call "thunk".
> >
> > The above logic allows clean timely finalization for
> > *multiple* ressources:
> >
> >     exec autoclose() with f1=open(name1), f2=open(name2, 'w'):
> >         for line in f1:
> >             ...
> >             f2.write(...)
> 
> That looks messy.

as compared to?  I think i have seen messier ideas with all kinds
of brackets and double-colons :-)
 
> > which would execute as follows
> >
> >     a) autoclose() instance is created and stored as the 
> >        "thunk"-handler
> 
> We need a name for these.  I've been using "monitor" for a while, but
> I'm not sure it's that apposite.

So far i'd liked "execution handler" because 'enter/exit/except' are IMHO
execution events which you can hook into.  Do you agree with calling
them 'execution events' or how would you call them? 

> >     b) f1/f2 are stored as attributes on the autoclose instance
> >
> >     c) f1/f2 are put into the local/global namespace (and nested ones
> >        if rebinding is allowed)
> >
> >     d) thunk executes (for line ...)
> >
> >     e) autoclose 'leave' hook is called (with or without exception)
> >        and is implemented like this:
> >         
> >        def __leave__(self):
> >             for obj in self.__dict__.values(): 
> >                 obj.close()
> >
> >     f) thunk handler is removed 
> 
> "Too much magic!"

Hmmm. That seems a bit unfair to me as many other proposals didn't care 
to elaborate the exact sequence.   You probably are refering to the 
"namespace interactions" as the other stuff is probably the same
with any other proposal so far.
 
> > Because computing 'f1' may succeed but 'f2' can subsequently
> > fail the assignments *have to* execute within "autoclose"
> > control.  

Would the following be the correct way to achieve this with your patch? 

    f1=open(inputfn)
    with autoclose(f1):
        f2 = open(outputfn, 'w')
        with autoclose(f2):
            for line in f1:
                ...
                f2.write(line)

I think there should be a better solution for multiple ressources.

> >
> > Now on to the usage of the except hook.  Nice use cases might be
> >
> >     exec retry(maxretry=3, on=IOError): 
> >         # do network io-stuff
> > or
> >     exec skip_on(AttributeError, TypeError):
> >         some_object.notify_hook()
> >
> > but i am sure there are more.  Exception handling is often
> > ugly when inlined with the code.  I think that stating 
> > 'exception behaviour' up-front allows to write nice 
> > readable constructs. 
> 
> I am *still* not convinced that an __except__ hook is worth the pain.
> Can you implement those for me?

Maybe, i haven't looked at your implementation yet and there are
some other projects pending :-)

> > __exit__ versus __leave__
> > ---------------------------
> >
> > One remark (mainly to Michael as he does that other 
> > patch) about the hook-name __leave__ versus __exit__.
> > we may want to eventually allow 'yield' within the
> > thunk and then '__exit__' would be misleading.
> 
> I guess.  But enter/exit is just such a canonical pairing.  To me,
> leave is paired with arrive and arrive/leave doesn't make much sense.
> 
> > Here is the use case:
> >
> >     exec self.mylock:   # would lock/unlock on entering/leaving 
> >                         # the generator
> >         ...
> >         for whatever in something:
> >             yield whatever  
> >         ...
> 
> yields already can't go in blocks with finally statements, right?

right. 

> Would you propose calling the __enter__ method each time the generator
> resumed?

yes, neccessary for locking (as the above example tried to indicate).

> > Or do you think that this (future) use case warrants 
> > yet another hook?
> 
> Guess :)

Then i guess it doesn't.  That way my point, anyway :-)

> > If there is interest i can probably modify my patch 
> > to allow all of the proposed syntax so that you could 
> > play around with it.  
> >
> > the next additional idea is not essential for my so-far
> > proposal (but hopefully interesting, nonetheless).
> 
> I think all these proposals are in danger of bending the language into
> a place it should not go.  Still, we should write some PEPs, even if
> they wind up rejected.

sure.

cheers,

    holger