A "scopeguard" for Python

Robert Kern robert.kern at gmail.com
Thu Mar 4 12:20:20 EST 2010


On 2010-03-04 10:56 AM, Alf P. Steinbach wrote:
> * Robert Kern:
>> On 2010-03-03 18:49 PM, Alf P. Steinbach wrote:
>>> * Robert Kern:
> [snip]
>>>> can you
>>>> understand why we might think that you were saying that try: finally:
>>>> was wrong and that you were proposing that your code was equivalent to
>>>> some try: except: else: suite?
>>>
>>> No, not really. His code didn't match the semantics. Changing 'finally'
>>> to 'else' could make it equivalent.
>>
>> Okay, please show me what you mean by "changing 'finally' to 'else'."
>> I think you are being hinty again. It's not helpful.
> [snip middle of this paragraph]
>> Why do you think that we would interpret those words to mean that you
>> wanted the example you give just above?
>
> There's an apparent discrepancy between your call for an example and
> your subsequent (in the same paragraph) reference to the example given.
>
> But as to why I assumed that that example, or a similar correct one,
> would be implied, it's the only meaningful interpretation.
>
> Adopting a meaningless interpretation when a meaningful exists is
> generally just adversarial, but in this case I was, as you pointed out,
> extremely unclear, and I'm sorry: I should have given such example up
> front. Will try to do so.

Thank you. I appreciate it.

> [snip]
>>
>>>> There are a couple of ways to do this kind of cleanup depending on the
>>>> situation. Basically, you have several different code blocks:
>>>>
>>>> # 1. Record original state.
>>>> # 2. Modify state.
>>>> # 3. Do stuff requiring the modified state.
>>>> # 4. Revert to the original state.
>>>>
>>>> Depending on where errors are expected to occur, and how the state
>>>> needs to get modified and restored, there are different ways of
>>>> arranging these blocks. The one Mike showed:
>>>>
>>>> # 1. Record original state.
>>>> try:
>>>> # 2. Modify state.
>>>> # 3. Do stuff requiring the modified state.
>>>> finally:
>>>> # 4. Revert to the original state.
>>>>
>>>> And the one you prefer:
>>>>
>>>> # 1. Record original state.
>>>> # 2. Modify state.
>>>> try:
>>>> # 3. Do stuff requiring the modified state.
>>>> finally:
>>>> # 4. Revert to the original state.
>>>>
>>>> These differ in what happens when an error occurs in block #2, the
>>>> modification of the state. In Mike's, the cleanup code runs; in yours,
>>>> it doesn't. For chdir(), it really doesn't matter. Reverting to the
>>>> original state is harmless whether the original chdir() succeeds or
>>>> fails, and chdir() is essentially atomic so if it raises an exception,
>>>> the state did not change and nothing needs to be cleaned up.
>>>>
>>>> However, not all block #2s are atomic. Some are going to fail partway
>>>> through and need to be cleaned up even though they raised an
>>>> exception. Fortunately, cleanup can frequently be written to not care
>>>> whether the whole thing finished or not.
>>>
>>> Yeah, and there are some systematic ways to handle these things. You
>>> might look up Dave Abraham's levels of exception safety. Mostly his
>>> approach boils down to making operations effectively atomic so as to
>>> reduce the complexity: ideally, if an operation raises an exception,
>>> then it has undone any side effects.
>>>
>>> Of course it can't undo the launching of an ICBM, for example...
>>>
>>> But ideally, if it could, then it should.
>>
>> I agree. Atomic operations like chdir() help a lot. But this is
>> Python, and exceptions can happen in many different places. If you're
>> not just calling an extension module function that makes a
>> known-atomic system call, you run the risk of not having an atomic
>> operation.
>>
>>> If you call the possibly failing operation "A", then that systematic
>>> approach goes like this: if A fails, then it has cleaned up its own
>>> mess, but if A succeeds, then it's the responsibility of the calling
>>> code to clean up if the higher level (multiple statements) operation
>>> that A is embedded in, fails.
>>>
>>> And that's what Marginean's original C++ ScopeGuard was designed for,
>>> and what the corresponding Python Cleanup class is designed for.
>>
>> And try: finally:, for that matter.
>
> Not to mention "with".
>
> Some other poster made the same error recently in this thread; it is a
> common fallacy in discussions about programming, to assume that since
> the same can be expressed using lower level constructs, those are all
> that are required.
>
> If adopted as true it ultimately means the removal of all control
> structures above the level of "if" and "goto" (except Python doesn't
> have "goto").

What I'm trying to explain is that the with: statement has a use even if Cleanup 
doesn't. Arguing that Cleanup doesn't improve on try: finally: does not mean 
that the with: statement doesn't improve on try: finally:.

>>>> Both formulations can be correct (and both work perfectly fine with
>>>> the chdir() example being used). Sometimes one is better than the
>>>> other, and sometimes not. You can achieve both ways with either your
>>>> Cleanup class or with try: finally:.
>>>>
>>>> I am still of the opinion that Cleanup is not an improvement over try:
>>>> finally: and has the significant ugliness of forcing cleanup code into
>>>> callables. This significantly limits what you can do in your cleanup
>>>> code.
>>>
>>> Uhm, not really. :-) As I see it.
>>
>> Well, not being able to affect the namespace is a significant
>> limitation. Sometimes you need to delete objects from the namespace in
>> order to ensure that their refcounts go to zero and their cleanup code
>> gets executed.
>
> Just a nit (I agree that a lambda can't do this, but as to what's
> required): assigning None is sufficient for that[1].

Yes, but no callable is going to allow you to assign None to names in that 
namespace, either. Not without sys._getframe() hackery, in any case.

> However, note that the current language doesn't guarantee such cleanup,
> at least as far as I know.
>
> So while it's good practice to support it, to do everything to let it
> happen, it's presumably bad practice to rely on it happening.
>
>
>> Tracebacks will keep the namespace alive and all objects in it.
>
> Thanks!, I hadn't thought of connecting that to general cleanup actions.
>
> It limits the use of general "with" in the same way.

Not really. It's easy to write context managers that do that. You put the 
initialization code in the __enter__() method, assign whatever objects you want 
to keep around through the with: clause as attributes on the manager, then 
delete those attributes in the __exit__(). Or, you use the @contextmanager 
decorator to turn a generator into a context manager, and you just assign to 
local variables and del them in the finally: clause.

What you can't do is write a generic context manager where the initialization 
happens inside the with: clause and the cleanup actions are registered 
callables. That does not allow you to affect the namespace.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco




More information about the Python-list mailing list