[Python-ideas] A different kind of context manager
Nick Coghlan
ncoghlan at gmail.com
Tue Oct 22 05:31:41 CEST 2013
On 22 Oct 2013 07:27, "Haoyi Li" <haoyi.sg at gmail.com> wrote:
>
> > Maybe we should have selected one in which this sort of coding is its
native, natural, form, rather than having this intermarriage kludge which
turns an imperative-looking generator into the traditional context manager.
>
> I agree with this 100%. Unfortunately, Python picked the current style of
`with` statements a while ago, and this would be a pretty huge
change/additional feature.
>
> FWIW, other languages with easy anonymous functions (e.g. ruby, scala)
have this, and it does provide all the benefits that you describe. For
example, spinning off parallel tasklets inline is just a matter of `async{
... }` where `async` is just a context manager.
It's not a coincidence that Ruby (at least - I don't know scala) just
treats for loops and context management as special cases of anonymous
callbacks - the latter is a powerful, more general construct.
By contrast, Python chose the path of providing dedicated syntax for both
iteration *and* context management, and hence requires that callbacks that
don't fit in a single expression be defined prior to use.
I think this makes those constructs easier to understand in many ways, but
it *also* means that we *don't* currently have a clean syntax for single
use callbacks.
Hence the time I've put into PEP 403 and 3150 over the years - a key
objective for both of them is providing a cleaner solution for the problem
of single use callbacks (including those that modify local variables of the
containing function).
In addition to scoping, the other problem single use callbacks need to
handle sensibly is the behaviour of the simple flow control statements:
return, yield, break, continue, and raise.
Building single use callbacks on top of the "def" statement has the
advantage of *not* needing to define any new scoping or control flow
semantics (as they're just ordinary nested scopes).
Defining them any other way makes things far more complicated. It would
certainly be close to impossible to repurpose any of the other existing
compound statements without breaking backwards compatibility.
A completely new keyword is also a possibility, but then it's necessary to
find a good one, and explain it's use cases in a fashion similar to PEP 403.
Cheers,
Nick.
> As much as I'd like to see python follow suite, getting it into python
would be a pretty large upheaval and not something that I expect to happen
in the near future.
>
> And the last option, if you're crazy, is to use MacroPy and write your
own tasklet/forking context managers! It's surprisingly easy (< 50 lines).
>
>
> On Mon, Oct 21, 2013 at 2:16 PM, Kristján Valur Jónsson <
kristjan at ccpgames.com> wrote:
>>
>> Ø Cool, sure. But what are the use cases that need this and can't be
done easily with the existing design?
>>
>> Exactly those I listed. Any form of execution that requires a
“function” to be run. This include all existing threading/multiprocessing
designs.
>>
>>
>>
>> Ø How would making the code inside the with block a callable improve
this?
>>
>> It allows the “if” statement to be part of the context manager
>>
>>
>>
>> Ø I think this code is easier to read than yours as the logic of
whether or not the do_stuff_once block is executed is where it belongs --
not hidden in the context manager.
>>
>> That is a matter of taste. Maybe it would make sense to pull other
things out of the context manager too? But taste should not, IMHO, limit
our options. If this is a common idiom, locking and executing once, why
shouldn’t we be able to write a clever macro/context manger/syntactic sugar
to help us with that? Why insist on this verbosity?
>>
>>
>>
>> Also, consider my argument that most context managers are written using
the @contextmanager paradigm. We like this idiom so much because this is
what we really want to do, call the code from a wrapper function.
>>
>> If you look at the design of that context manager, it is not exactly
straightforward. This suggests to me that maybe we took a wrong turn
deciding on a context manager design. Maybe we should have selected one in
which this sort of coding is its native, natural, form, rather than having
this intermarriage kludge which turns an imperative-looking generator into
the traditional context manager.
>>
>>
>>
>>
>>
>> From: Bruce Leban [mailto:bruce at leapyear.org]
>> Sent: 21. október 2013 17:22
>> To: Kristján Valur Jónsson
>> Cc: python-ideas at python.org
>>
>>
>> Subject: Re: [Python-ideas] A different kind of context manager
>>
>>
>>
>>
>>
>> On Mon, Oct 21, 2013 at 6:55 AM, Kristján Valur Jónsson <
kristjan at ccpgames.com> wrote:
>>
>> So, If this is the way people like to think about context managers, like
writing wrapper functoins, why don‘t we turn them into proper wrapper
functions?
>>
>> <...> The cool thing here though, is that "code" could, for example, be
run on a different tasklet. Or a different thread. Or a different
universe.
>>
>> Cool, sure. But what are the use cases that need this and can't be done
easily with the existing design?
>>
>>
>>>
>>> class NewContextManager(object):
>>>
>>> # A context manager that locks a resource, then executes the code
only if it is not recursing
>>> def __init__(self, lock):
>>> self.lock = lock
>>> def __contextcall__(self, code):
>>> with lock:
>>> if lock.active:
>>> return # This is where @contextmanager will stop you, you
can’t skip the ‘yield’
>>> lock.active = True
>>> try:
>>> return code(None) # optionally pass value to the code as in
"with foo() as X"
>>> finally:
>>> lock.active = False
>>
>>
>>
>> You can do that with current context managers:
>>
>>
>>>
>>> with lock_once(x) as lock_acquired:
>>>
>>> if lock_acquired: # was not already locked
>>>
>>> do_stuff_once()
>>>
>>>
>>>
>>> @contextmanager
>>>
>>> def lock_once(lock):
>>>
>>> if lock.active:
>>>
>>> yield False
>>>
>>> else:
>>>
>>> lock.active = True
>>>
>>> try:
>>>
>>> yield True
>>>
>>> finally:
>>>
>>> lock.active = False
>>
>>
>>
>> Note that I'm mimicking your lock/unlock code which of course is not the
proper way to acquire/release a lock, but it gets the idea across. How
would making the code inside the with block a callable improve this? I
think this code is easier to read than yours as the logic of whether or not
the do_stuff_once block is executed is where it belongs -- not hidden in
the context manager. Note that my version also allows me to do this, which
I can't easily do with your context manager:
>>
>>
>>>
>>> with lock_once(x) as lock_acquired:
>>>
>>> if lock_acquired: # was not already locked
>>>
>>> do_stuff_once()
>>>
>>> else:
>>>
>>> log('Lock %r was already acquired', x)
>>>
>>> do_stuff_every_time()
>>
>>
>>
>>
>>
>> --- Bruce
>> I'm hiring: http://www.cadencemd.com/info/jobs
>>
>> Latest blog post: Alice's Puzzle Page http://www.vroospeak.com
>>
>> Learn how hackers think: http://j.mp/gruyere-security
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131022/099538f1/attachment.html>
More information about the Python-ideas
mailing list