revisit pep 377: good use case?
So I've recently been trying to implement something for which I had hoped the 'with' statement would be perfect, but it turns out, won't work because Python provides no mechanism by which to skip the block of code in a with statement. I want to create some functionality to make it easy too wrap command line programs in a caching architecture. To do this there are some things that need to happen before and after the wrapped CLI program is called, a try,except,finally version might look like this: def cachedcli(*args): try: hashedoutput = hashon(args) if iscached(): return hashedoutput acquirelock() cli(*args,hashedoutput) iscached(True) return hashedoutput except AlreadyLocked: while locked: wait() return example(*args) finally: releaselock() the 'with' version would look like def cachedcli(*args) hashedpath = hashon(args) with cacheon(hashedpath): cli(hashedpath,*args) return hashedpath So obviously the 'with' statement would be a good fit, especially since non-python programmers might be wrapping their CLI programs... unfortunately I can't use 'with' because I can't find a clean way to make the with block code conditional. PEP377 suggested some mechanics that seemed a bit complicated for getting the desired effect, but I think, and correct me if I'm wrong, that the same effect could be achieved by having the __enter__ function raise a StopIteration that would be caught by the context and skip directly to the __exit__ function. The semantics of this even make some sense too me, since the closest I've been able to get to what I had hoped for was using an iterator to execute the appropriate code before and after the loop block: def cachedcli(*args) hashedpath = hashon(args) for _ in cacheon(hashedpath): cli(hashedpath,*args) return hashedpath this still seems non-ideal to me...
On Wed, Feb 29, 2012 at 5:02 PM, Craig Yoshioka <craigyk@me.com> wrote:
PEP377 suggested some mechanics that seemed a bit complicated for getting the desired effect, but I think, and correct me if I'm wrong, that the same effect could be achieved by having the __enter__ function raise a StopIteration that would be caught by the context and skip directly to the __exit__ function.
It was the overhead of doing exception handling around the __enter__ call that got PEP 377 rejected. One way to handle this case is to use a separate if statement to make the flow control clear. with cm() as run_body: if run_body: # Do stuff Depending on the use case, the return value from __enter__ may be a simple flag as shown, or it may be a more complex object. Alternatively, you may want to investigate contextlib2, which aims to provide improved support for conditional cleanup in with statements. (in the current version, this is provided by contextlib2.ContextStack, but the next version will offer an improved API as contextlib2.CallbackStack. No current ETA on the next update though) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 29 February 2012 08:23, Nick Coghlan <ncoghlan@gmail.com> wrote:
PEP377 suggested some mechanics that seemed a bit complicated for getting the desired effect, but I think, and correct me if I'm wrong, that
On Wed, Feb 29, 2012 at 5:02 PM, Craig Yoshioka <craigyk@me.com> wrote: the same effect could be achieved by having the __enter__ function raise a StopIteration that would be caught by the context and skip directly to the __exit__ function.
It was the overhead of doing exception handling around the __enter__ call that got PEP 377 rejected.
One way to handle this case is to use a separate if statement to make the flow control clear.
with cm() as run_body: if run_body: # Do stuff
Depending on the use case, the return value from __enter__ may be a simple flag as shown, or it may be a more complex object.
The trouble with this is it indents all your code an extra level. One possibility would be allowing continue in a with statement as an early exit: with cm() as run_body: if not run_body: continue Michael
Alternatively, you may want to investigate contextlib2, which aims to provide improved support for conditional cleanup in with statements. (in the current version, this is provided by contextlib2.ContextStack, but the next version will offer an improved API as contextlib2.CallbackStack. No current ETA on the next update though)
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On Wed, Feb 29, 2012 at 6:24 AM, Michael Foord <fuzzyman@gmail.com> wrote:
On 29 February 2012 08:23, Nick Coghlan <ncoghlan@gmail.com> wrote:
One way to handle this case is to use a separate if statement to make the flow control clear.
with cm() as run_body: if run_body: # Do stuff
Depending on the use case, the return value from __enter__ may be a simple flag as shown, or it may be a more complex object.
The trouble with this is it indents all your code an extra level. One possibility would be allowing continue in a with statement as an early exit:
with cm() as run_body: if not run_body: continue
-1 on this as an early __exit__. It would be context dependent. For with statements within a loop body, a continue today continues to the next loop iteration. Introducing this syntax would call into question what continue does... exit the with statement within the loop body? or continue the loop (also exiting the with statement but skipping all other code in the loop body)? for x in range(5): with y() as run_body: if not run_body: continue print x, run_body Changing the existing continue semantics would break existing code and adding continue semantics to exit a context manager that care if a with is within a loop body or not seems very unwise and surprising. -gps
On Wed, Feb 29, 2012 at 09:02, Craig Yoshioka <craigyk@me.com> wrote:
So I've recently been trying to implement something for which I had hoped the 'with' statement would be perfect, but it turns out, won't work because Python provides no mechanism by which to skip the block of code in a with statement.
I want to create some functionality to make it easy too wrap command line programs in a caching architecture. To do this there are some things that need to happen before and after the wrapped CLI program is called, a try,except,finally version might look like this:
def cachedcli(*args): try: hashedoutput = hashon(args) if iscached(): return hashedoutput acquirelock() cli(*args,hashedoutput) iscached(True) return hashedoutput except AlreadyLocked: while locked: wait() return example(*args) finally: releaselock()
the 'with' version would look like
def cachedcli(*args) hashedpath = hashon(args) with cacheon(hashedpath): cli(hashedpath,*args) return hashedpath
So obviously the 'with' statement would be a good fit, especially since non-python programmers might be wrapping their CLI programs... unfortunately I can't use 'with' because I can't find a clean way to make the with block code conditional.
PEP377 suggested some mechanics that seemed a bit complicated for getting the desired effect, but I think, and correct me if I'm wrong, that the same effect could be achieved by having the __enter__ function raise a StopIteration that would be caught by the context and skip directly to the __exit__ function. The semantics of this even make some sense too me, since the closest I've been able to get to what I had hoped for was using an iterator to execute the appropriate code before and after the loop block:
def cachedcli(*args) hashedpath = hashon(args) for _ in cacheon(hashedpath): cli(hashedpath,*args) return hashedpath
this still seems non-ideal to me...
Specifically with regard to caching, I recommend writing a CLI execution class which implements the caching logic internally. If you really want to do this with some special syntax sugar, use decorators, which are good for wrapping functions/methods with caching. The "with" statement is IMO not suitable here (and rightfully so). - Tal Einat
I've tried classes, decorators, and passing the conditional using 'as', as suggested by Michael, so I disagree that with is not suitable here since I have yet to find a better alternative. If you want I can give pretty concrete examples in the ways they aren't as good. Furthermore, I think it could be argued that it makes more sense to be able to safely skip the with body without the user of the with statement having to manually catch the exception themselves.... we don't make people catch the StopIteration exception manually when using iterators... 1) I can't think of many instances in python where a block of code can not be conditionally executed safely: if - obvious functions - need to be called loops - can have 0 or more iterations try/except/finally - even here there is the same notion of the code blocks being conditionally executed, just a bit more scrambled in my view, the 'with' statement exists just because it is nice sugar for bracketing boilerplate around a block of code, so it might as well do that in the most general, reasonable way. And I think this behavior is pretty reasonable. On Feb 29, 2012, at 06:07 AM, Tal Einat <taleinat@gmail.com> wrote: On Wed, Feb 29, 2012 at 09:02, Craig Yoshioka <craigyk@me.com> wrote:
So I've recently been trying to implement something for which I had hoped the 'with' statement would be perfect, but it turns out, won't work because Python provides no mechanism by which to skip the block of code in a with statement.
I want to create some functionality to make it easy too wrap command line programs in a caching architecture. To do this there are some things that need to happen before and after the wrapped CLI program is called, a try,except,finally version might look like this:
def cachedcli(*args): try: hashedoutput = hashon(args) if iscached(): return hashedoutput acquirelock() cli(*args,hashedoutput) iscached(True) return hashedoutput except AlreadyLocked: while locked: wait() return example(*args) finally: releaselock()
the 'with' version would look like
def cachedcli(*args) hashedpath = hashon(args) with cacheon(hashedpath): cli(hashedpath,*args) return hashedpath
So obviously the 'with' statement would be a good fit, especially since non-python programmers might be wrapping their CLI programs... unfortunately I can't use 'with' because I can't find a clean way to make the with block code conditional.
PEP377 suggested some mechanics that seemed a bit complicated for getting the desired effect, but I think, and correct me if I'm wrong, that the same effect could be achieved by having the __enter__ function raise a StopIteration that would be caught by the context and skip directly to the __exit__ function. The semantics of this even make some sense too me, since the closest I've been able to get to what I had hoped for was using an iterator to execute the appropriate code before and after the loop block:
def cachedcli(*args) hashedpath = hashon(args) for _ in cacheon(hashedpath): cli(hashedpath,*args) return hashedpath
this still seems non-ideal to me...
Specifically with regard to caching, I recommend writing a CLI execution class which implements the caching logic internally. If you really want to do this with some special syntax sugar, use decorators, which are good for wrapping functions/methods with caching. The "with" statement is IMO not suitable here (and rightfully so). - Tal Einat
Craig Yoshioka wrote:
I've tried classes, decorators, and passing the conditional using 'as', as suggested by Michael, so I disagree that with is not suitable here since I have yet to find a better alternative. If you want I can give pretty concrete examples in the ways they aren't as good.
I would be interested in your concrete examples. As far as conditionally skipping the with body, where would that code go? In __enter__? How would it know whether or not to skip? ~Ethan~
Ok, I'll go clean them up to try and present them as concisely as possible. The code to skip the with body would have to go in the __enter__ method because wether the body should be executed is dependent on the semantics of the context being used. Imagine a context that looks like: with uncached('file') as file: # write data to file Making the context skippable only from __enter__ means the person writing the context can be more confident of the possible code paths. And the person writing the body code could always just 'skip' manually anyways by returning early, i.e. per Michael's suggestion. with uncached('file') as file: if not file: return which isn't so bad, except it is overloading the meaning of file a bit, and why shouldn't the with block be skippable? I can see a couple of ways it might work: 1) catch raised StopIteration, or a new 'SkipWithBlock', exception thrown from the __enter__ code 2) skip the with block when __enter__ returns a unique value like SkipWithBlock, otherwise assign the returned value using 'as' In my mind 2 should be easy? to implement, and shouldn't break any existing code since the new sentinel value didn't exist before anyways. Maybe it would be more efficient that also wrapping __enter__ in yet another try|except|finally? On Feb 29, 2012, at 11:29 AM, Ethan Furman wrote:
Craig Yoshioka wrote:
I've tried classes, decorators, and passing the conditional using 'as', as suggested by Michael, so I disagree that with is not suitable here since I have yet to find a better alternative. If you want I can give pretty concrete examples in the ways they aren't as good.
I would be interested in your concrete examples.
As far as conditionally skipping the with body, where would that code go? In __enter__? How would it know whether or not to skip?
~Ethan~
Craig Yoshioka wrote:
Ok, I'll go clean them up to try and present them as concisely as possible. The code to skip the with body would have to go in the __enter__ method because wether the body should be executed is dependent on the semantics of the context being used. Imagine a context that looks like:
with uncached('file') as file: # write data to file
Making the context skippable only from __enter__ means the person writing the context can be more confident of the possible code paths. And the person writing the body code could always just 'skip' manually anyways by returning early, i.e. per Michael's suggestion.
with uncached('file') as file: if not file: return
Can you give an example of the code that would be in __enter__? ~Ethan~
On Feb 29, 2012, at 11:55 AM, Ethan Furman wrote:
From PEP 343:
But the final blow came when I read Raymond Chen's rant about flow-control macros[1]. Raymond argues convincingly that hiding flow control in macros makes your code inscrutable, and I find that his argument applies to Python as well as to C.
So it is explicitly stated that the with statement should not be capable of controlling the flow.
I read the rant, and I agree in principle, but I think it's also a far stretch to draw a line between a very confusing non-standard example of macros in C, and documentable behavior of a built-in statement. That is, the only reason you might say with would be hiding flow-control is because people don't currently expect it to. I also think that when people use non-builtin contextmanagers it's usually within a very specific... context (*dammit*), and so they are likely to look up why they are using an object as a context manager. That's where you would document the behavior: with uncached(path): # code here only executes if the path does not exist
Indeed.
Craig, if you want to pursue this to the extent of writing up a full PEP, I suggest starting with the idea I briefly wrote up a while ago [1].
Instead of changing the semantics of __enter__, add a new optional method __entered__ to the protocol that executes inside the with statement's implicit try/except block.
That is (glossing over the complexities in the real with statement expansion), something roughly like:
_exit = cm.__exit__ _entered = getattr(cm, "__entered__", None) _var = cm.__enter__() try: if _entered is not None: _var = _entered(_var) VAR = _var # if 'as' clause is present # with statement body finally: _exit(*sys.exc_info())
that is an interesting alternative... do you see that as much better than __enter__ passing some sort of unique value to signal the skip? I can't say I'm enamored of doing it with a signal value, just thought it would be easier to implement (and not require more exception handling): _exit = cm.__exit__ _var = cm.__enter__() if _var == SkipWithBody: _exit(None,None,None) try: VAR = _var # if 'as' clause is present # with statement body finally: _exit(*sys.exc_info())
Craig Yoshioka wrote:
On Feb 29, 2012, at 11:55 AM, Ethan Furman wrote:
From PEP 343:
But the final blow came when I read Raymond Chen's rant about flow-control macros[1]. Raymond argues convincingly that hiding flow control in macros makes your code inscrutable, and I find that his argument applies to Python as well as to C.
So it is explicitly stated that the with statement should not be capable of controlling the flow.
I read the rant, and I agree in principle, but I think it's also a far stretch to draw a line between a very confusing non-standard example of macros in C, and documentable behavior of a built-in statement. That is, the only reason you might say with would be hiding flow-control is because people don't currently expect it to. I also think that when people use non-builtin contextmanagers it's usually within a very specific... context (*dammit*), and so they are likely to look up why they are using an object as a context manager. That's where you would document the behavior:
with uncached(path): # code here only executes if the path does not exist
I am -1 on the idea. if / while / for / try are *always* flow control. Your proposal would have 'with' sometimes being flow control, and sometimes not, and the only way to know is look at the object's code and/or docs. This makes for a lot more complication for very little gain. ~Ethan~
On Feb 29, 2012 4:56 PM, "Ethan Furman" <ethan@stoneleaf.us> wrote:
Craig Yoshioka wrote:
On Feb 29, 2012, at 11:55 AM, Ethan Furman wrote:
From PEP 343:
But the final blow came when I read Raymond Chen's rant about flow-control macros[1]. Raymond argues convincingly that hiding flow control in macros makes your code inscrutable, and I find that his argument applies to Python as well as to C.
So it is explicitly stated that the with statement should not be capable of controlling the flow.
I read the rant, and I agree in principle, but I think it's also a far
stretch to draw a line between a very confusing non-standard example of macros in C, and documentable behavior of a built-in statement. That is, the only reason you might say with would be hiding flow-control is because people don't currently expect it to. I also think that when people use non-builtin contextmanagers it's usually within a very specific... context (*dammit*), and so they are likely to look up why they are using an object as a context manager. That's where you would document the behavior:
with uncached(path): # code here only executes if the path does not exist
I am -1 on the idea.
if / while / for / try are *always* flow control.
Your proposal would have 'with' sometimes being flow control, and sometimes not, and the only way to know is look at the object's code and/or docs. This makes for a lot more complication for very little gain.
~Ethan~
I like the general idea, but a conditionally conditional control syntax is a readability nightmare., however, I wonder if the case in which the with statement act as a conditional could be explicit so a reader can distinguish between those that will always execute their body and those which may or may not. with cached(key): do_caching() else: update_exp(key) _______________________________________________
Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On 29 February 2012 21:47, Craig Yoshioka <craigyk@me.com> wrote:
I also think that when people use non-builtin contextmanagers it's usually within a very specific... context (*dammit*), and so they are likely to look up why they are using an object as a context manager. That's where you would document the behavior:
with uncached(path): # code here only executes if the path does not exist
Personally, I would *always* assume that the body of the with statement executes. That's what the with statement does, and I would be very surprised to see something different happen. Even with the comment, I'd be surprised. Paul.
On Wed, Feb 29, 2012 at 6:08 PM, Paul Moore <p.f.moore@gmail.com> wrote:
Personally, I would *always* assume that the body of the with statement executes. That's what the with statement does, and I would be very surprised to see something different happen. Even with the comment, I'd be surprised.
with open('/path/that/doesnt/exist.txt', 'r') as f: # code that doesn't get executed pass -- Devin
Ethan Furman wrote: re-posting to list Craig Yoshioka wrote:
Here is what the context might look like:
class Uncached(object): def __init__(self,path): self.path = path self.lock = path + '.locked' def __enter__(self): if os.path.exists(self.path): return SKipWithBlock # skips body goes straight to __exit__ try: os.close(os.open(self.lock,os.O_CREAT|os.O_EXCL|os.O_RDWR)) except OSError as e: if e.errno != errno.EEXIST: raise while os.path.exists(self.lock): time.sleep(0.1) return self.__enter__() return self.path def __exit__(self,et,ev,st): if os.path.exists(self.lock): os.unlink(self.lock)
class Cache(object): def __init__(self,*args,**kwargs): self.base = os.path.join(CACHE_DIR,hashon(args,kwargs)) #..... def create(self,path): return Uncached(os.path.join(self.base,path)) #.....
def cached(func): def wrapper(*args,**kwargs): cache = Cache(*args,**kwargs) return func(cache,*args,**kwargs) return wrapper
--------------------------------------------------------------------- Person using code: ---------------------------------------------------------------------
@cached def createdata(cache,x): path = cache.pathfor('output.data') with cache.create(path) as cpath: with open(cpath,'wb') as cfile: cfile.write(x*10000) return path
pool.map(createdata,['x','x','t','x','t'])
---------------------------------------------------------------------
so separate processes return the path to the cached data and create it if it doesn't exist, and even wait if another process is working on it.
my collaborators could hopefully very easily wrap their programs with minimal effort using the cleanest syntax possible, and since inputs get hashed to consistent output paths for each wrapped function, the wrapped functions can be easily combined, chained, etc. and behind the scenes they are reusing as much work as possible.
Here are the current possible alternatives:
1. use the passed var as a flag, they must insert the if for every use of the context, if not, then cached results get recomputed
@cached def createdata(cache,x): path = cache.pathfor('output.data') with cache.create(path) as cpath: if not cpath: return with open(cpath,'wb') as cfile: cfile.write(x*10000) return path
2. using the for loop and an iterator instead of a context, is more fool-proof, but a bit confusing?
@cached def createdata(cache,x): path = cache.pathfor('output.data') for cpath in cache.create(path): if not cpath: return with open(cpath,'wb') as cfile: cfile.write(x*10000) return path
3. using a class the outputs and caching function need to be specified separately so that calls can be scripted together, also a lot more boilerplate:
class createdata(CachedWrapper): def outputs(self,x): self.outputs += [self.cache.pathfor('output.data')] def tocache(self,x): with open(self.outputs[0],'wb') as cfile: cfile.write(x*10000)
Craig Yoshioka wrote:
with uncached('file') as file: if not file: return
which isn't so bad, except it is overloading the meaning of file a bit, and
No, I don't agree with that. It is normal Pythonic idiom for objects to be interpreted in a boolean concept. The only difference here is that sometimes file will be (presumably) a file-like object, and sometimes it needs to be a sentinel like None. But this shouldn't disturb you: you have already suggested one possible implementation would be for __enter__ to return a special value, SkipWithBlock, to force skipping the block. That's fine: you can have this functionality *right now*. All you need do is change the spelling from SkipWithBlock to None, and use an explicit test inside the block rather than an implicit test, and you're done. Best of all, the use of an explicit test means you can do this: with whatever('file') as file: if file is None: log('error') else: normal_processing() which you can't do with your suggested implicit test-and-skip. The fact that with blocks are not flow-control is a feature, not a bug.
why shouldn't the with block be skippable?
It is skippable. Like every other code block, it is skippable by wrapping it in flow-control code to decide whether or not to skip it. -- Steven
The fact that with blocks are not flow-control is a feature, not a bug.
why shouldn't the with block be skippable?
It is skippable. Like every other code block, it is skippable by wrapping it in flow-control code to decide whether or not to skip it.
I'm not claiming the current functionality is a bug, just unfortunate.
On 29 February 2012 17:49, Craig Yoshioka <craigyk@me.com> wrote:
I've tried classes, decorators, and passing the conditional using 'as', as suggested by Michael, so I disagree that with is not suitable here since I have yet to find a better alternative. If you want I can give pretty concrete examples in the ways they aren't as good. Furthermore, I think it could be argued that it makes more sense to be able to safely skip the with body without the user of the with statement having to manually catch the exception themselves....
From PEP 343:
But the final blow came when I read Raymond Chen's rant about flow-control macros[1]. Raymond argues convincingly that hiding flow control in macros makes your code inscrutable, and I find that his argument applies to Python as well as to C. So it is explicitly stated that the with statement should not be capable of controlling the flow. -- Arnaud [1] Raymond Chen's article on hidden flow control http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx
On Thu, Mar 1, 2012 at 6:29 AM, Arnaud Delobelle <arnodel@gmail.com> wrote:
On 29 February 2012 17:49, Craig Yoshioka <craigyk@me.com> wrote:
I've tried classes, decorators, and passing the conditional using 'as', as suggested by Michael, so I disagree that with is not suitable here since I have yet to find a better alternative. If you want I can give pretty concrete examples in the ways they aren't as good. Furthermore, I think it could be argued that it makes more sense to be able to safely skip the with body without the user of the with statement having to manually catch the exception themselves....
From PEP 343:
But the final blow came when I read Raymond Chen's rant about flow-control macros[1]. Raymond argues convincingly that hiding flow control in macros makes your code inscrutable, and I find that his argument applies to Python as well as to C.
So it is explicitly stated that the with statement should not be capable of controlling the flow.
Indeed. Craig, if you want to pursue this to the extent of writing up a full PEP, I suggest starting with the idea I briefly wrote up a while ago [1]. Instead of changing the semantics of __enter__, add a new optional method __entered__ to the protocol that executes inside the with statement's implicit try/except block. That is (glossing over the complexities in the real with statement expansion), something roughly like: _exit = cm.__exit__ _entered = getattr(cm, "__entered__", None) _var = cm.__enter__() try: if _entered is not None: _var = _entered(_var) VAR = _var # if 'as' clause is present # with statement body finally: _exit(*sys.exc_info()) Then CM's would be free to skip directly from __entered__ to __exit__ by raising a custom exception. GeneratorContextManagers could similarly be updated to handle the case where the underlying generator doesn't yield. However, that last point highlights why I no longer like the idea: it makes it *really* easy to accidentally create CM's that, instead of throwing an exception if you try to reuse them inappropriately, will instead silently skip the with statement body. The additional expressiveness provided by such a construct is minimal, but the additional risk of incorrectly silencing errors is quite high - that's not a good trade-off for the overall language design. [1] http://readthedocs.org/docs/ncoghlan_devs-python-notes/en/latest/pep_ideas/s... Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Craig Yoshioka wrote:
I've tried classes, decorators, and passing the conditional using 'as', as suggested by Michael, so I disagree that with is not suitable here since I have yet to find a better alternative. If you want I can give pretty concrete examples in the ways they aren't as good. Furthermore, I think it could be argued that it makes more sense to be able to safely skip the with body without the user of the with statement having to manually catch the exception themselves.... we don't make people catch the StopIteration exception manually when using iterators...
Sometimes we do. When you call next(it) manually, you are responsible for catching the exception manually. It is only flow-control tools (e.g. for loops, list comprehensions) that catch the exception for you.
1) I can't think of many instances in python where a block of code can not be conditionally executed safely: if - obvious functions - need to be called loops - can have 0 or more iterations try/except/finally - even here there is the same notion of the code blocks being conditionally executed, just a bit more scrambled
Classes are blocks of code. If you want to conditionally skip executing all, or part, of a class block you wrap it in an if block. (Or try...except.) This is as it should be: the class statement is not a flow control statement. Nor is the def statement. In this case, the right concept is not in *calling* the function body, but in *compiling* the function body. Again, if you want to conditionally skip compiling all or part of the function body, you have to wrap it in a flow control structure, if or try. We don't have any concept of: class K: body def f(): body where something inside the bodies invisibly determines whether or not the K block executes or the f block compiles. -- Steven
On Feb 29, 2012, at 4:00 PM, Steven D'Aprano wrote:
Sometimes we do. When you call next(it) manually, you are responsible for catching the exception manually. It is only flow-control tools (e.g. for loops, list comprehensions) that catch the exception for you.
I know that. I'm arguing that 'with' could catch the for loop equivalent of StopIteration. Or at least that's one implementation.
Classes are blocks of code. If you want to conditionally skip executing all, or part, of a class block you wrap it in an if block. (Or try...except.) This is as it should be: the class statement is not a flow control statement.
Nor is the def statement. In this case, the right concept is not in *calling* the function body, but in *compiling* the function body. Again, if you want to conditionally skip compiling all or part of the function body, you have to wrap it in a flow control structure, if or try.
We don't have any concept of:
class K: body
def f(): body
where something inside the bodies invisibly determines whether or not the K block executes or the f block compiles.
? I'm just making an observation that there aren't many other places where an 'independent' block is 'guaranteed' to run. I'm not arguing that something inside a block is magically preventing execution of the block. The with keyword is 'outside' the block. From within a block you can always return/break early anyways. Think about it: def test(): # block is only executed if function is called if True: # block is only executed if True for x in items: # block is executed only for each item class T(object): def m(self): # if method called but I find it surprising that all of a sudden it's correct to assume: with x: # it's obvious this block will always run that was not my assumption when I first became aware of 'with'. For me it only meant that the indented block ran within some sort of context that was being managed for me. Not saying the current use is wrong (though I do find it inconvenient and inconsistent), just that, other than 'with' I'm trying to think of a situation in which I'd put code in a new block where that code is guaranteed to run. I suppose I assumed that contexts were fancier if statements that were guaranteed to create and clean up various aspects of state.
-- Steven _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On Thu, Mar 1, 2012 at 10:46 AM, Craig Yoshioka <craigyk@me.com> wrote:
? I'm just making an observation that there aren't many other places where an 'independent' block is 'guaranteed' to run. I'm not arguing that something inside a block is magically preventing execution of the block. The with keyword is 'outside' the block. From within a block you can always return/break early anyways.
There are two other cases where a suite is guaranteed to at least start executing, and one of them is the only case that matters here: "try" blocks. The flow control behaviour of "with:" is consistent with the behaviour of a "try:" block, and that is not an accident. Conditional execution of a try block requires a separate if statement or throwing an exception that is caught and suppressed by one of the exception handlers. Similarly, conditional execution of a with block requires a separate if statement or throwing an exception that is caught and suppressed by one of the context managers. Hence, arguments of language consistency aren't going to get you anywhere here. Guido's verdict on PEP 377 was that the payoff (avoiding a separate if statement or a method call that deliberately throws an appropriate exception in certain niche use cases) wasn't worth the additional complexity in the with statement definition. I now agree with him. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I agree about 'try' being another example of a, at least not immediately, flow-controlled block, I think I mentioned as much in a different response. But you're not going to win me over by saying that the #1 goal of 'with' is too best match the flow-control semantics of try|except|finally. The #1 goal, I would have hoped, was to abstract away the pattern of using try:except:finally to manage a block of code that does IO stuff. And the current 'with' doesn't even perfectly match the semantics of try|except|finally because if it did, then 'with' uses of try|except|finally that I can think of that don't have a with equivalent. try: # boilerplate while True: # boilerplate if alreadyCached(): # boilerplate break # boilerplate try: # boilerplate acquireLock() # boilerplate doSomeStuff() # <- code that does doMoreStuff() # <- stuff goes here alreadyCached(True) # boilerplate except AlreadyLocked: # boilerplate sleep() # boilerplate finally: # boilerplate cleanUpMyLockIfItExists() # boilerplate There isn't a 'with' equivalent to the above that hides all the unnecessary context state and boilerplate from the enclosed block, therefore the 'client' code must know and remember to check the appropriate flags everytime and if they don't the context's functionality may be broken. if not alreadyCached(): # <- check before we execute context block with aquireCache() as cache: ... # <- to run the ... # code they put in here with aquireCache() as cache: # <- slightly better than above if not cache: # <- but may still 'break' the context if they forget this ... # <- to run the ... # code they put in here On Feb 29, 2012, at 6:15 PM, Nick Coghlan wrote:
There are two other cases where a suite is guaranteed to at least start executing, and one of them is the only case that matters here: "try" blocks. The flow control behaviour of "with:" is consistent with the behaviour of a "try:" block, and that is not an accident.
Conditional execution of a try block requires a separate if statement or throwing an exception that is caught and suppressed by one of the exception handlers. Similarly, conditional execution of a with block requires a separate if statement or throwing an exception that is caught and suppressed by one of the context managers. Hence, arguments of language consistency aren't going to get you anywhere here.
Guido's verdict on PEP 377 was that the payoff (avoiding a separate if statement or a method call that deliberately throws an appropriate exception in certain niche use cases) wasn't worth the additional complexity in the with statement definition. I now agree with him.
Regards, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (11)
-
Arnaud Delobelle -
Calvin Spealman -
Craig Yoshioka -
Devin Jeanpierre -
Ethan Furman -
Gregory P. Smith -
Michael Foord -
Nick Coghlan -
Paul Moore -
Steven D'Aprano -
Tal Einat