[Python-ideas] A different kind of context manager

[Kristján Valur Jónsson]

Hello there!
This is a rehash of something that I wrote on stackless-dev recently.  Perhaps this has been suggested in the past.  If so, please excuse my ignorance.

It irks me sometimes how inflexible context managers can be.  For example, wouldn't it be nice to be able to write
with as_subprocess():
    do_stuff()

or in StacklessPython:
with different_tasklet():
  do_stuff()

This is currently impossible because context managers are implemented as __enter__()/__exit__() methods running in the current scope.
There is no callable function site to pass ot a subprocess module, or a tasklet scheduler.


I have another favorite pet-peeve, which is that for some things, a pair of context managers are needed, since a single context manager cannot silence it's own exception:
  with IgnoreError, LockResourceOrRaiseIfBusy(resource):
    do_stuff

cannot be collapsed into:
  with LockResourceOrPassIfBusy(resource):
    do_stuff.


But another thing is also interesting:  Even though context managers are an __enter__() / __exit__() pair, the most common idiom these days is to write:
@contextlib.contextmanager
def mycontextmanager():
  setup()
  try:
    yield
  finally():
    teardown()

or similar.
There are a million reasons for this.  Mostly it is because this layout is easier to figure out and plays nicer in the head.  It also simplifies error handling, because regular try/except clauses can be used.  If you are writing a "raw" context manager, you have to explicitly maintain some state between the __enter__() and __exit__() methods to know what to clean up and how, depending on the error conditions.  This quickly becomes tedious.

And what does the above code look like?  Well, the place of the "yield", could just as well be a call site.  I mean, the decorated "contextmanager" function simply looks like a wrapper function around a function call.  You write it exactly as you would write a wrapper function ,except where you would call the function, you use the "yield" statement (and you _have_ to call yield.  Can't skip it for whatever reason).

So, If this is the way people like to think about context managers, like writing wrapper functoins, why don't we turn them into proper wrapper functions?

What if a context manager were given a _callable_, representing the code?

like this:



class NewContextManager(object):

  # A context manager that locks a resource, then executes the code only if it is not recursing

  def __init__(self, lock):

    self.lock = lock

  def __contextcall__(self, code):

    with lock:

      if lock.active:

        return  # This is where @contextmanager will stop you, you can't skip the 'yield'

      lock.active = True

      try:

        return code(None) # optionally pass value to the code as in "with foo() as X"

      finally:

        lock.active = False





The cool thing here though, is that "code" could, for example, be run on a different tasklet.  Or a different thread.  Or a different universe:

def TaskletContextManager(object):

  def __contextcall__(self, code):

    return stacklesslib.run_on_tasklet(code)



def ThreadContextManager(object):

  def __contextcall__(self, code):

    result = []

    def helper():

      result.append(code())

    t = threading.Thread(target=helper)

    t.start()

    t.join()

    return result[0]





This sort of thing would need compiler and syntax support, of course.  The compiler would need to create an anonymous function object.   The return value out of "code" would be some token that could be special if the code returned....



To illustrate, let's see how this can be done manually:
This code here:
  with foo() as bar :
     if condition:
        return stuff
      do_stuff(bar)

can be re-written like this:

  def _code(_arg):
    bar = _arg
    if condition:
      return True, stuff  # early return
   do_stuff(bar)
   return False, None # no return

  token, value = foo(bar).__contextcall__(_code):
  if token is True
   return value

where:
  class foo(object):
    def __init__(self, arg):
      self.arg = arg
    def __contextcall__(self, _code):
   set_up()
    try:
      return _code(None) #pass it some value
    finally:
      tear_down()







Compiler support for this sort of thing would entail the automatic creation of the "_code" function as an anonymous function with special semantics for a "return" value.  This function is then passed to the __contextcall__() method of the "new" context manager, where the context manager treats it as any other callable, which' return value it must return.



The "early return" can also be done as a special kind of exception, ContextManagerReturn(value).



So, anyway.  Context manager syntax is really nice for so many reasons, which is why we have it in the language, instead of wrapper functions.  But if it _were_ just syntactic sugar for actual wrapper functions, they would be even awesomer.



K





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131021/fe78a029/attachment-0001.html>