I'd like to be able to do this: with *context_managers: pass # Some suite. This is useful when you have an unknown number of context managers that you want to use. I currently use `contextlib.nested`, but I'd like the *star syntax much better. What do you think? Ram.
Il 01 aprile 2012 22:25, Ram Rachum <ram.rachum@gmail.com> ha scritto:
I believe writing a specialized context manager object which is able to hold multiple context managers altogheter is better than introducing a new syntax for such a use case which should be pretty rare/uncommon. Also, it's not clear what to expect from "with *context_managers as ctx: ...". Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/
On 2 April 2012 12:40, Giampaolo Rodolà <g.rodola@gmail.com> wrote:
There's now an example of a need for this in the standard library. mock.patch collects together an arbitrary number of context managers that need to be entered sequentially (together). As there is no replacement for contextlib.nested it has custom code calling __enter__ and __exit__ on all the context managers and keeping track of which ones have been successfully entered (because if there is an exception whilst entering one, only those that have *already* been entered should have __exit__ called).
Also, it's not clear what to expect from "with *context_managers as ctx: ...".
It is clear. It should be a tuple of results (what else *could* it be). Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
Ram Rachum schrieb am Sun, 01. Apr 2012, um 13:25:41 -0700:
'contextlib.nested()' is broken and not available in Python 3.x (the language this list is about). The only replacement so far is with CM1() as cm1, CM2() as cm2: ... which only works for a fixed number of context managers. And there is a class 'ContextStack' in Nick Coghlan's 'contextlib2' library [1], which might be included in Python 3.3. With this class, you could write your code as with ContextStack() as stack: for cm in context_managers: stack.enter_context(cm) This still leaves the question whether your proposed syntax would be preferable, also with regard to issue 2292 [2]. [1]: http://readthedocs.org/docs/contextlib2/en/latest/#contextlib2.ContextStack [2]: http://bugs.python.org/issue2292 Cheers, Sven
On Tue, Apr 3, 2012 at 1:34 AM, Sven Marnach <sven@marnach.net> wrote:
Both "with *(iterable)" and "for cm in iterable: stack.enter(cm)" are flawed in exactly the same way that contextlib.nested() is flawed: they encourage creating the iterable of context managers first, which means that inner __init__ methods are not covered by outer __exit__ methods. This breaks as soon as you have resources (such as files) where the acquire/release resource management pairing is actually __init__/__exit__ with __enter__ just returning self rather than acquiring the resource. If the iterable of context managers is created first, then the outer resources *will be leaked* if any of the inner constructors fail. The only way to write code that handles an arbitrary number of arbitrary context managers in a robust fashion is to ensure the initialisation steps are also covered by the outer context managers: with CallbackStack() as stack: for make_cm in cm_factories: stack.enter(make_cm()) (Note that I'm not particularly happy with the class and method names for contextlib2.ContextStack, and plan to redesign it a bit before adding it to the stdlib module: https://bitbucket.org/ncoghlan/contextlib2/issue/8/rename-contextstack-to-ca...) The only time you can get away with a contextlib.nested() style API where the iterable of context managers is created first is when you *know* that all of the context managers involved do their resource acquisition in __enter__ rather than __init__. In the general case, though, any such API is broken because it doesn't reliably clean up files and similar acquired-on-initialisation resources. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Another proposal, change nested to work with generators: with nested(open(path) for path in files): .... pros: 1. lazy evaluation of context manager creation (which is everything that is bad with today's nested). 2. shorter than ContextStack. cons: 1. generators are not always an option, in these cases ContextStack is the way to go. 2. a little implicit - I can imagine a python newbie swearing my mom because he didn't know he should use a generator instead of a list. thanks, Alon Horev On Mon, Apr 2, 2012 at 11:52 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Il 01 aprile 2012 22:25, Ram Rachum <ram.rachum@gmail.com> ha scritto:
I believe writing a specialized context manager object which is able to hold multiple context managers altogheter is better than introducing a new syntax for such a use case which should be pretty rare/uncommon. Also, it's not clear what to expect from "with *context_managers as ctx: ...". Regards, --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/
On 2 April 2012 12:40, Giampaolo Rodolà <g.rodola@gmail.com> wrote:
There's now an example of a need for this in the standard library. mock.patch collects together an arbitrary number of context managers that need to be entered sequentially (together). As there is no replacement for contextlib.nested it has custom code calling __enter__ and __exit__ on all the context managers and keeping track of which ones have been successfully entered (because if there is an exception whilst entering one, only those that have *already* been entered should have __exit__ called).
Also, it's not clear what to expect from "with *context_managers as ctx: ...".
It is clear. It should be a tuple of results (what else *could* it be). Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
Ram Rachum schrieb am Sun, 01. Apr 2012, um 13:25:41 -0700:
'contextlib.nested()' is broken and not available in Python 3.x (the language this list is about). The only replacement so far is with CM1() as cm1, CM2() as cm2: ... which only works for a fixed number of context managers. And there is a class 'ContextStack' in Nick Coghlan's 'contextlib2' library [1], which might be included in Python 3.3. With this class, you could write your code as with ContextStack() as stack: for cm in context_managers: stack.enter_context(cm) This still leaves the question whether your proposed syntax would be preferable, also with regard to issue 2292 [2]. [1]: http://readthedocs.org/docs/contextlib2/en/latest/#contextlib2.ContextStack [2]: http://bugs.python.org/issue2292 Cheers, Sven
On Tue, Apr 3, 2012 at 1:34 AM, Sven Marnach <sven@marnach.net> wrote:
Both "with *(iterable)" and "for cm in iterable: stack.enter(cm)" are flawed in exactly the same way that contextlib.nested() is flawed: they encourage creating the iterable of context managers first, which means that inner __init__ methods are not covered by outer __exit__ methods. This breaks as soon as you have resources (such as files) where the acquire/release resource management pairing is actually __init__/__exit__ with __enter__ just returning self rather than acquiring the resource. If the iterable of context managers is created first, then the outer resources *will be leaked* if any of the inner constructors fail. The only way to write code that handles an arbitrary number of arbitrary context managers in a robust fashion is to ensure the initialisation steps are also covered by the outer context managers: with CallbackStack() as stack: for make_cm in cm_factories: stack.enter(make_cm()) (Note that I'm not particularly happy with the class and method names for contextlib2.ContextStack, and plan to redesign it a bit before adding it to the stdlib module: https://bitbucket.org/ncoghlan/contextlib2/issue/8/rename-contextstack-to-ca...) The only time you can get away with a contextlib.nested() style API where the iterable of context managers is created first is when you *know* that all of the context managers involved do their resource acquisition in __enter__ rather than __init__. In the general case, though, any such API is broken because it doesn't reliably clean up files and similar acquired-on-initialisation resources. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Another proposal, change nested to work with generators: with nested(open(path) for path in files): .... pros: 1. lazy evaluation of context manager creation (which is everything that is bad with today's nested). 2. shorter than ContextStack. cons: 1. generators are not always an option, in these cases ContextStack is the way to go. 2. a little implicit - I can imagine a python newbie swearing my mom because he didn't know he should use a generator instead of a list. thanks, Alon Horev On Mon, Apr 2, 2012 at 11:52 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
participants (8)
-
Alon Horev -
Antoine Pitrou -
Eric V. Smith -
Giampaolo Rodolà -
Michael Foord -
Nick Coghlan -
Ram Rachum -
Sven Marnach