A "scopeguard" for Python

[Alf P. Steinbach]

* Robert Kern:
> On 2010-03-03 18:49 PM, Alf P. Steinbach wrote:
>> * Robert Kern:
[snip]
>>>  can you
>>> understand why we might think that you were saying that try: finally:
>>> was wrong and that you were proposing that your code was equivalent to
>>> some try: except: else: suite?
>>
>> No, not really. His code didn't match the semantics. Changing 'finally'
>> to 'else' could make it equivalent.
> 
> Okay, please show me what you mean by "changing 'finally' to 'else'." I 
> think you are being hinty again. It's not helpful.  
[snip middle of this paragraph]
> Why do you think that we would interpret those words 
> to mean that you wanted the example you give just above?

There's an apparent discrepancy between your call for an example and your 
subsequent (in the same paragraph) reference to the example given.

But as to why I assumed that that example, or a similar correct one, would be 
implied, it's the only meaningful interpretation.

Adopting a meaningless interpretation when a meaningful exists is generally just 
adversarial, but in this case I was, as you pointed out, extremely unclear, and 
I'm sorry: I should have given such example up front. Will try to do so.


[snip]
> 
>>> There are a couple of ways to do this kind of cleanup depending on the
>>> situation. Basically, you have several different code blocks:
>>>
>>> # 1. Record original state.
>>> # 2. Modify state.
>>> # 3. Do stuff requiring the modified state.
>>> # 4. Revert to the original state.
>>>
>>> Depending on where errors are expected to occur, and how the state
>>> needs to get modified and restored, there are different ways of
>>> arranging these blocks. The one Mike showed:
>>>
>>> # 1. Record original state.
>>> try:
>>> # 2. Modify state.
>>> # 3. Do stuff requiring the modified state.
>>> finally:
>>> # 4. Revert to the original state.
>>>
>>> And the one you prefer:
>>>
>>> # 1. Record original state.
>>> # 2. Modify state.
>>> try:
>>> # 3. Do stuff requiring the modified state.
>>> finally:
>>> # 4. Revert to the original state.
>>>
>>> These differ in what happens when an error occurs in block #2, the
>>> modification of the state. In Mike's, the cleanup code runs; in yours,
>>> it doesn't. For chdir(), it really doesn't matter. Reverting to the
>>> original state is harmless whether the original chdir() succeeds or
>>> fails, and chdir() is essentially atomic so if it raises an exception,
>>> the state did not change and nothing needs to be cleaned up.
>>>
>>> However, not all block #2s are atomic. Some are going to fail partway
>>> through and need to be cleaned up even though they raised an
>>> exception. Fortunately, cleanup can frequently be written to not care
>>> whether the whole thing finished or not.
>>
>> Yeah, and there are some systematic ways to handle these things. You
>> might look up Dave Abraham's levels of exception safety. Mostly his
>> approach boils down to making operations effectively atomic so as to
>> reduce the complexity: ideally, if an operation raises an exception,
>> then it has undone any side effects.
>>
>> Of course it can't undo the launching of an ICBM, for example...
>>
>> But ideally, if it could, then it should.
> 
> I agree. Atomic operations like chdir() help a lot. But this is Python, 
> and exceptions can happen in many different places. If you're not just 
> calling an extension module function that makes a known-atomic system 
> call, you run the risk of not having an atomic operation.
> 
>> If you call the possibly failing operation "A", then that systematic
>> approach goes like this: if A fails, then it has cleaned up its own
>> mess, but if A succeeds, then it's the responsibility of the calling
>> code to clean up if the higher level (multiple statements) operation
>> that A is embedded in, fails.
>>
>> And that's what Marginean's original C++ ScopeGuard was designed for,
>> and what the corresponding Python Cleanup class is designed for.
> 
> And try: finally:, for that matter.

Not to mention "with".

Some other poster made the same error recently in this thread; it is a common 
fallacy in discussions about programming, to assume that since the same can be 
expressed using lower level constructs, those are all that are required.

If adopted as true it ultimately means the removal of all control structures 
above the level of "if" and "goto" (except Python doesn't have "goto").


>>> Both formulations can be correct (and both work perfectly fine with
>>> the chdir() example being used). Sometimes one is better than the
>>> other, and sometimes not. You can achieve both ways with either your
>>> Cleanup class or with try: finally:.
>>>
>>> I am still of the opinion that Cleanup is not an improvement over try:
>>> finally: and has the significant ugliness of forcing cleanup code into
>>> callables. This significantly limits what you can do in your cleanup
>>> code.
>>
>> Uhm, not really. :-) As I see it.
> 
> Well, not being able to affect the namespace is a significant 
> limitation. Sometimes you need to delete objects from the namespace in 
> order to ensure that their refcounts go to zero and their cleanup code 
> gets executed.

Just a nit (I agree that a lambda can't do this, but as to what's required): 
assigning None is sufficient for that[1].

However, note that the current language doesn't guarantee such cleanup, at least 
as far as I know.

So while it's good practice to support it, to do everything to let it happen, 
it's presumably bad practice to rely on it happening.


> Tracebacks will keep the namespace alive and all objects 
> in it.

Thanks!, I hadn't thought of connecting that to general cleanup actions.

It limits the use of general "with" in the same way.


Cheers,

- Alf

Notes:
[1] An 'except' clause deletes variables, but since it has no knowledge of the 
code it's placed in the only alternatives would be a presumably costly check of 
prior existence, or letting it pollute the namespace.