[Python-Dev] 'With' context documentation draft (was Re: Terminology for PEP 343

[Nick Coghlan]

M.-A. Lemburg wrote:
> This is exactly what I'm getting at: I can see the potential
> use for resource management (which is what started out the
> whole idea IIRC), but fail to see why you'd want to use it
> for anything more complicated than that.

The other suggested uses (redirecting stdout, logging exceptions, inserting 
fences in markup, switching decimal context) aren't really more complicated. 
They're just different cases of 'paired code' - code which consists of 
'standard setup code', 'the unique bit', 'standard cleanup code'.

The longest example context manager in PEP 343 is only 11 lines long. Even the 
wrapper to turn a generator into a context manager weighs in at only 23 lines.

> Once you start talking about contexts (which usually refers
> to a combination of environment, state and location) you
> have to explain things like nesting, scopes, combination
> of different contexts, their interaction with each other,
> etc. etc.

Except that context managers combine contexts in a very simple way - each one 
is almost entirely independent of both its surroundings and its contents. Each 
context manager takes a specific action to set the context up, and a specific 
action to tear it down again afterwards. Its behaviour is dependent only on 
the arguments that are passed to its constructor, and exceptions raised by its 
contents.

The only way to make a context manager depend on its position in the code 
(rather than the passed in arguments) is to use sys._getframe, and using that 
is always considered an evil hack :)

> Note that hiding things away in smart objects like what
> you call "context managers" will not make programs easier
> to understand, unless the specific task that such a "context
> manager" is simple enough to grasp by just looking at its
> definition in the with statement... but then you'd not call
> it a "context manager".

Each one on its own may not be called a context manager, but we still need a 
collective term that covers all of the simple (but different) cases of common 
setup and cleanup code that we would like to factor out.

We tried 'resource managers' (which was too specific to one particular use 
case), 'block managers' (a bit too close to the related-but-very-different 
Ruby blocks) and 'suite managers' (which wasn't too bad, but not very 
intuitive) along the way, but 'context managers' was the first one that really 
seemed to adequately summarise the nature of the protocol - allowing a 
programmer to factor out arbitrary setup and cleanup code that would otherwise 
have to be reproduced inline everywhere they needed it.

> BTW, the same argument applies to decorators. Programs don't
> get easier to read or understand if you overload a function
> with lots and lots of hidden magic...

No argument here, but notice that the generic term 'decorator' tells you 
nothing about what any *specific* decorator does. The obviousness needs to be 
in the name of the decorator (such as 'classmethod'). The situation is the 
same for context managers - to make the code easy to read, the method or 
variable used to denote the context manager in the with statement must be well 
named.

> @put_all_the_smart_stuff_here
> def program(context):
>     return "42"
> 
> Of course, you *could* do all these things and Python is
> not preventing you from doing so, but will life get easier ?
> I doubt it.

But what about:

   for item in put_all_the_smart_stuff_here():
       item.value = 42

The other guiding light in the terminology discussion was the existing concept 
of iterables and iterators. Those can do just about anything you want, and 
frequently hide a great deal of complexity. However, if they're well named, 
then it is possible to read a for loop that uses them without batting an eyelid.

If the reader decides they need to know the details, they look up the 
documentation of the iterator, just as they'd look up the details of a 
decorator they didn't recognise. Once they've looked up the definition once, 
instead of having to recognise a particular usage pattern in the code ("Oh, 
that's doing 'X' again"), the reader simply has to recognise the name ("Oh, 
that's the decorator/iterator/context manager that does 'X'")

> Let's keep things simple and Python nice.

Indeed - but I see being able to factor out common code as both simple *and* 
nice, which is why I like the ideas of functions, iterators, decorators and 
context managers :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.blogspot.com