[Python-Dev] PEP 377 - allow __enter__() methods to skip the statement body
Michael Foord
fuzzyman at voidspace.org.uk
Sun Mar 15 18:50:15 CET 2009
Brett Cannon wrote:
> Without knowing what StatementSkipped is (just some singleton? If so
> why not just used SkipStatement instance that was raised?) and
> wondering if we are just going to continue to adding control flow
> exceptions that directly inherit from BaseException or some
> ControlFlowException base class, the basic idea seems fine by me.
>
Note that using exceptions for control flow can be bad for other
implementations of Python. For example exceptions on the .NET framework
are very expensive. (Although there are workarounds such as not really
raising the exception - but they're ugly).
Isn't it better practise for exceptions to be used for exceptional
circumstances rather than for control flow?
Michael
> On Sun, Mar 15, 2009 at 05:56, Nick Coghlan <ncoghlan at gmail.com
> <mailto:ncoghlan at gmail.com>> wrote:
>
> PEP 377 is a proposal to allow context manager __enter__() methods to
> skip the body of the with statement by raising a specific (new) flow
> control exception.
>
> Since there is a working reference implementation now, I thought
> it was
> time to open it up for broader discussion.
>
> Full PEP attached, or you can find it in the usual place at
> http://www.python.org/dev/peps/pep-0377
>
> Cheers,
> Nick.
>
> P.S. I expect a rationale for the StatementSkipped value binding is
> probably going to be pretty high on the list of questions that aren't
> currently covered by the PEP. I hope to write more on that some time
> this week.
>
> --
> Nick Coghlan | ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>
> | Brisbane, Australia
> ---------------------------------------------------------------
>
> PEP: 377
> Title: Allow __enter__() methods to skip the statement body
> Version: $Revision: 70384 $
> Last-Modified: $Date: 2009-03-15 22:48:49 +1000 (Sun, 15 Mar 2009) $
> Author: Nick Coghlan <ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 8-Mar-2009
> Python-Version: 2.7, 3.1
> Post-History: 8-Mar-2009
>
>
> Abstract
> ========
>
> This PEP proposes a backwards compatible mechanism that allows
> ``__enter__()``
> methods to skip the body of the associated ``with`` statement. The
> lack of
> this ability currently means the ``contextlib.contextmanager``
> decorator
> is unable to fulfil its specification of being able to turn arbitrary
> code into a context manager by moving it into a generator function
> with a yield in the appropriate location. One symptom of this is that
> ``contextlib.nested`` will currently raise ``RuntimeError`` in
> situations where writing out the corresponding nested ``with``
> statements would not [1].
>
> The proposed change is to introduce a new flow control exception
> ``SkipStatement``, and skip the execution of the ``with``
> statement body if ``__enter__()`` raises this exception.
>
>
> Proposed Change
> ===============
>
> The semantics of the ``with`` statement will be changed to include a
> new ``try``/``except``/``else`` block around the call to
> ``__enter__()``.
> If ``SkipStatement`` is raised by the ``__enter__()`` method, then
> the main section of the ``with`` statement (now located in the
> ``else``
> clause) will not be executed. To avoid leaving the names in any ``as``
> clause unbound in this case, a new ``StatementSkipped`` singleton
> (similar to the existing ``NotImplemented`` singleton) will be
> assigned to all names that appear in the ``as`` clause.
>
> The components of the ``with`` statement remain as described in
> PEP 343 [2]::
>
> with EXPR as VAR:
> BLOCK
>
> After the modification, the ``with`` statement semantics would
> be as follows::
>
> mgr = (EXPR)
> exit = mgr.__exit__ # Not calling it yet
> try:
> value = mgr.__enter__()
> except SkipStatement:
> VAR = StatementSkipped
> # Only if "as VAR" is present and
> # VAR is a single name
> # If VAR is a tuple of names, then StatementSkipped
> # will be assigned to each name in the tuple
> else:
> exc = True
> try:
> try:
> VAR = value # Only if "as VAR" is present
> BLOCK
> except:
> # The exceptional case is handled here
> exc = False
> if not exit(*sys.exc_info()):
> raise
> # The exception is swallowed if exit() returns true
> finally:
> # The normal and non-local-goto cases are handled here
> if exc:
> exit(None, None, None)
>
> With the above change in place for the ``with`` statement semantics,
> ``contextlib.contextmanager()`` will then be modified to raise
> ``SkipStatement`` instead of ``RuntimeError`` when the underlying
> generator doesn't yield.
>
>
> Rationale for Change
> ====================
>
> Currently, some apparently innocuous context managers may raise
> ``RuntimeError`` when executed. This occurs when the context
> manager's ``__enter__()`` method encounters a situation where
> the written out version of the code corresponding to the
> context manager would skip the code that is now the body
> of the ``with`` statement. Since the ``__enter__()`` method
> has no mechanism available to signal this to the interpreter,
> it is instead forced to raise an exception that not only
> skips the body of the ``with`` statement, but also jumps over
> all code until the nearest exception handler. This goes against
> one of the design goals of the ``with`` statement, which was to
> be able to factor out arbitrary common exception handling code
> into a single context manager by putting into a generator
> function and replacing the variant part of the code with a
> ``yield`` statement.
>
> Specifically, the following examples behave differently if
> ``cmB().__enter__()`` raises an exception which ``cmA().__exit__()``
> then handles and suppresses::
>
> with cmA():
> with cmB():
> do_stuff()
> # This will resume here without executing "do_stuff()"
>
> @contextlib.contextmanager
> def combined():
> with cmA():
> with cmB():
> yield
>
> with combined():
> do_stuff()
> # This will raise a RuntimeError complaining that the context
> # manager's underlying generator didn't yield
>
> with contextlib.nested(cmA(), cmB()):
> do_stuff()
> # This will raise the same RuntimeError as the contextmanager()
> # example (unsurprising, given that the nested() implementation
> # uses contextmanager())
>
> # The following class based version shows that the issue isn't
> # specific to contextlib.contextmanager() (it also shows how
> # much simpler it is to write context managers as generators
> # instead of as classes!)
> class CM(object):
> def __init__(self):
> self.cmA = None
> self.cmB = None
>
> def __enter__(self):
> if self.cmA is not None:
> raise RuntimeError("Can't re-use this CM")
> self.cmA = cmA()
> self.cmA.__enter__()
> try:
> self.cmB = cmB()
> self.cmB.__enter__()
> except:
> self.cmA.__exit__(*sys.exc_info())
> # Can't suppress in __enter__(), so must raise
> raise
>
> def __exit__(self, *args):
> suppress = False
> try:
> if self.cmB is not None:
> suppress = self.cmB.__exit__(*args)
> except:
> suppress = self.cmA.__exit__(*sys.exc_info()):
> if not suppress:
> # Exception has changed, so reraise explicitly
> raise
> else:
> if suppress:
> # cmB already suppressed the exception,
> # so don't pass it to cmA
> suppress = self.cmA.__exit__(None, None, None):
> else:
> suppress = self.cmA.__exit__(*args):
> return suppress
>
> With the proposed semantic change in place, the contextlib based
> examples
> above would then "just work", but the class based version would need
> a small adjustment to take advantage of the new semantics::
>
> class CM(object):
> def __init__(self):
> self.cmA = None
> self.cmB = None
>
> def __enter__(self):
> if self.cmA is not None:
> raise RuntimeError("Can't re-use this CM")
> self.cmA = cmA()
> self.cmA.__enter__()
> try:
> self.cmB = cmB()
> self.cmB.__enter__()
> except:
> if self.cmA.__exit__(*sys.exc_info()):
> # Suppress the exception, but don't run
> # the body of the with statement either
> raise SkipStatement
> raise
>
> def __exit__(self, *args):
> suppress = False
> try:
> if self.cmB is not None:
> suppress = self.cmB.__exit__(*args)
> except:
> suppress = self.cmA.__exit__(*sys.exc_info()):
> if not suppress:
> # Exception has changed, so reraise explicitly
> raise
> else:
> if suppress:
> # cmB already suppressed the exception,
> # so don't pass it to cmA
> suppress = self.cmA.__exit__(None, None, None):
> else:
> suppress = self.cmA.__exit__(*args):
> return suppress
>
> There is currently a tentative suggestion [3] to add import-style
> syntax to
> the ``with`` statement to allow multiple context managers to be
> included in
> a single ``with`` statement without needing to use
> ``contextlib.nested``. In
> that case the compiler has the option of simply emitting multiple
> ``with``
> statements at the AST level, thus allowing the semantics of actual
> nested
> ``with`` statements to be reproduced accurately. However, such a
> change
> would highlight rather than alleviate the problem the current PEP
> aims to
> address: it would not be possible to use
> ``contextlib.contextmanager`` to
> reliably factor out such ``with`` statements, as they would
> exhibit exactly
> the same semantic differences as are seen with the ``combined()``
> context
> manager in the above example.
>
>
> Performance Impact
> ==================
>
> Implementing the new semantics makes it necessary to store the
> references
> to the ``__enter__`` and ``__exit__`` methods in temporary
> variables instead
> of on the stack. This results in a slight regression in ``with``
> statement
> speed relative to Python 2.6/3.1. However, implementing a custom
> ``SETUP_WITH`` opcode would negate any differences between the two
> approaches (as well as dramatically improving speed by eliminating
> more
> than a dozen unnecessary trips around the eval loop).
>
>
> Reference Implementation
> ========================
>
> Patch attached to Issue 5251 [1]. That patch uses only existing
> opcodes
> (i.e. no ``SETUP_WITH``).
>
>
> Acknowledgements
> ================
>
> James William Pye both raised the issue and suggested the basic
> outline of
> the solution described in this PEP.
>
>
> References
> ==========
>
> .. [1] Issue 5251: contextlib.nested inconsistent with nested with
> statements
> (http://bugs.python.org/issue5251)
>
> .. [2] PEP 343: The "with" Statement
> (http://www.python.org/dev/peps/pep-0343/)
>
> .. [3] Import-style syntax to reduce indentation of nested with
> statements
>
> (http://mail.python.org/pipermail/python-ideas/2009-March/003188.html)
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
> ..
> Local Variables:
> mode: indented-text
> indent-tabs-mode: nil
> sentence-end-double-space: t
> fill-column: 70
> End:
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <mailto:Python-Dev at python.org>
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog
More information about the Python-Dev
mailing list