[Python-ideas] with ... except

[Andrew Barnert]

> From: Nick Coghlan <ncoghlan at gmail.com>

> Sent: Saturday, March 9, 2013 6:34 AM

I agree completely with your main point. But:

> contextlib.ExitStack at least brings context managers close to on par
> with iterables - you can use either stack.enter_context(cm) and
> iter(iterable) to lift just the __enter__ or __iter__ call out into a
> separate try block. Wrapping an exception handler around next() pretty
> much requires reverting to a while loop, though.


It's worth noting off the bat that you don't actually need this too often, because most iterators can't be resumed after exception anyway. But when you do, it's possible. Of course you need a next-wrapper, but that's no different from needing a with-wrapper or an iter-wrapper. For example, let's say you wanted this fictitious construct:

    fooed_it = ((try: x.foo() except: continue) for x in it)


You can just do this:

    def skip_exceptions(it):

        while True:
            try:
                yield next(it)
            except StopIteration:
                raise
            except:
                pass


    fooed_it = (x.foo() for x in skip_exceptions(it))


And needless to say, you can put in a realistic exception handler instead of just a skip-everything clause.

I've actually got code similar to this. I've got a C-library enumerator-type function that can return both fatal and non-fatal errors. The first-level wrapper around this provides a next() that raises on non-fatal errors. Then I've got a wrapper around that which logs and continues for all exceptions but StopIteration and fatal errors.

> Ultimately, though, this may be an inevitable price we pay for the

> abstraction - you *do* lose flexibility when you design for the
> typical case, and so you do eventually have to say "sorry, to handle
> that more complex case you need to drop back down to the lower level
> syntax" (try/except/else/finally for with statements, while loops for
> for loops). 

I think it's even better than that. You can use the higher-level abstractions even in more complex cases, as long as you build a somewhat complex wrapper. And this has the same tradeoffs as any refactoring—if you save more complexity in the top-level code than you spend in the helper code, it's worth doing.

But the end result is the same: Making the abstractions more flexible makes them more complex, and there's a point at which the benefit (not requiring helpers in as many cases) loses to the cost.