Relax __exit__ method to be a generator
Literally it may be generator itself or function that returns generator object. Now I'm working on postgres library for asyncio (http://aiopg.readthedocs.org/). And I would to use *with statement* for transaction handling like: with (yield from cursor.transaction()): yield from cursor.execute(sql) The problem is: at exit of *with statement* I need to call `yield from cursor.execute('COMMIT')` or `yield from cursor.execute('ROLLBACK')`. I can do it only in __exit__ in *context manager*, but python understand only if __exit__: - returns true value, that suppresses exception from code block - returns None or any false value to propagate exception if any - raises exception itself I propose to add new rule: IF the code object is generator (co_flags & CO_GENERATOR) and __exit__ returns generator object (isinstance(ret, types.GeneratorType)) THEN do `yield from ret`. That's work fine if __exit__ itself is a *generator function* (contains `yield` or `yield from` statements): call to *generator function* returns *generator object*. The proposal: 1. Doesn't break any existing code except if user accidentally returns generator object instead of True from __exit__ call (he should not to do this and I sure this is very rare case). 2. Don't requires new syntax. asyncio itself uses: with (yield from lock): BLOCK for locking etc but unlocking for asyncio objects doesn't requires any `yield from`, so __exit__ code is just plain function but not generator. Also I can live with asyncio trick for __enter__: https://code.google.com/p/tulip/source/browse/asyncio/locks.py#156 The way is a but annoying but unrolling a value returned by __enter__ if the value is generator object will break existing code, sure. Thoughts? -- Thanks, Andrew Svetlov
I prefer to use a try/finally statement. One of the points of using yield-from is that you can always tell where your code may be suspended by searching for "yield from". With your proposed change that would no longer be true -- any with statement would also have to be inspected, and there would no longer be a way to know from the source alone whether it might yield or not (because it would dynamic -- there's no way to be sure at compile time which context manager is being used). On Sun, Apr 6, 2014 at 12:02 PM, Andrew Svetlov <andrew.svetlov@gmail.com>wrote:
Literally it may be generator itself or function that returns generator object.
Now I'm working on postgres library for asyncio ( http://aiopg.readthedocs.org/).
And I would to use *with statement* for transaction handling like:
with (yield from cursor.transaction()): yield from cursor.execute(sql)
The problem is: at exit of *with statement* I need to call `yield from cursor.execute('COMMIT')` or `yield from cursor.execute('ROLLBACK')`.
I can do it only in __exit__ in *context manager*, but python understand only if __exit__: - returns true value, that suppresses exception from code block - returns None or any false value to propagate exception if any - raises exception itself
I propose to add new rule: IF the code object is generator (co_flags & CO_GENERATOR) and __exit__ returns generator object (isinstance(ret, types.GeneratorType)) THEN do `yield from ret`.
That's work fine if __exit__ itself is a *generator function* (contains `yield` or `yield from` statements): call to *generator function* returns *generator object*.
The proposal: 1. Doesn't break any existing code except if user accidentally returns generator object instead of True from __exit__ call (he should not to do this and I sure this is very rare case). 2. Don't requires new syntax.
asyncio itself uses:
with (yield from lock): BLOCK
for locking etc but unlocking for asyncio objects doesn't requires any `yield from`, so __exit__ code is just plain function but not generator.
Also I can live with asyncio trick for __enter__: https://code.google.com/p/tulip/source/browse/asyncio/locks.py#156 The way is a but annoying but unrolling a value returned by __enter__ if the value is generator object will break existing code, sure.
Thoughts?
-- Thanks, Andrew Svetlov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
Well, this is good point.. But from my experience try/finally is a common point of errors. Beginners use "with" properly as: with open(filename) as f: f.read() but switch to try/finally produces very common error. Instead of f = open(filename) try: f.read() finally: f.close() people usually write try: f = open(filename) f.read() finally: f.close() I saw it constantly many times. When I wrote an article about the problem and true solution in my blog I got several emails from my readers: please, tell me again why I need to move open() out of try block if it can generate exception on file opening? So maybe you would like some other syntax? For example with from obj: BLOCK That form can use magic methods with names different than __enter__/__exit__ (and get rid of "with (yield from lock)" BTW). And it's obviously? points on two suspending points: at BLOCK enter and exit. On Mon, Apr 7, 2014 at 2:12 AM, Guido van Rossum <guido@python.org> wrote:
I prefer to use a try/finally statement. One of the points of using yield-from is that you can always tell where your code may be suspended by searching for "yield from". With your proposed change that would no longer be true -- any with statement would also have to be inspected, and there would no longer be a way to know from the source alone whether it might yield or not (because it would dynamic -- there's no way to be sure at compile time which context manager is being used).
On Sun, Apr 6, 2014 at 12:02 PM, Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
Literally it may be generator itself or function that returns generator object.
Now I'm working on postgres library for asyncio (http://aiopg.readthedocs.org/).
And I would to use *with statement* for transaction handling like:
with (yield from cursor.transaction()): yield from cursor.execute(sql)
The problem is: at exit of *with statement* I need to call `yield from cursor.execute('COMMIT')` or `yield from cursor.execute('ROLLBACK')`.
I can do it only in __exit__ in *context manager*, but python understand only if __exit__: - returns true value, that suppresses exception from code block - returns None or any false value to propagate exception if any - raises exception itself
I propose to add new rule: IF the code object is generator (co_flags & CO_GENERATOR) and __exit__ returns generator object (isinstance(ret, types.GeneratorType)) THEN do `yield from ret`.
That's work fine if __exit__ itself is a *generator function* (contains `yield` or `yield from` statements): call to *generator function* returns *generator object*.
The proposal: 1. Doesn't break any existing code except if user accidentally returns generator object instead of True from __exit__ call (he should not to do this and I sure this is very rare case). 2. Don't requires new syntax.
asyncio itself uses:
with (yield from lock): BLOCK
for locking etc but unlocking for asyncio objects doesn't requires any `yield from`, so __exit__ code is just plain function but not generator.
Also I can live with asyncio trick for __enter__: https://code.google.com/p/tulip/source/browse/asyncio/locks.py#156 The way is a but annoying but unrolling a value returned by __enter__ if the value is generator object will break existing code, sure.
Thoughts?
-- Thanks, Andrew Svetlov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
-- Thanks, Andrew Svetlov
Well, for new syntax, the bar is pretty high... And we should probably move to python-ideas. On Sun, Apr 6, 2014 at 4:44 PM, Andrew Svetlov <andrew.svetlov@gmail.com>wrote:
Well, this is good point..
But from my experience try/finally is a common point of errors. Beginners use "with" properly as:
with open(filename) as f: f.read()
but switch to try/finally produces very common error.
Instead of
f = open(filename) try: f.read() finally: f.close()
people usually write
try: f = open(filename) f.read() finally: f.close()
I saw it constantly many times. When I wrote an article about the problem and true solution in my blog I got several emails from my readers: please, tell me again why I need to move open() out of try block if it can generate exception on file opening?
So maybe you would like some other syntax? For example
with from obj: BLOCK
That form can use magic methods with names different than __enter__/__exit__ (and get rid of "with (yield from lock)" BTW). And it's obviously? points on two suspending points: at BLOCK enter and exit.
On Mon, Apr 7, 2014 at 2:12 AM, Guido van Rossum <guido@python.org> wrote:
I prefer to use a try/finally statement. One of the points of using yield-from is that you can always tell where your code may be suspended by searching for "yield from". With your proposed change that would no longer be true -- any with statement would also have to be inspected, and there would no longer be a way to know from the source alone whether it might yield or not (because it would dynamic -- there's no way to be sure at compile time which context manager is being used).
On Sun, Apr 6, 2014 at 12:02 PM, Andrew Svetlov < andrew.svetlov@gmail.com> wrote:
Literally it may be generator itself or function that returns generator object.
Now I'm working on postgres library for asyncio (http://aiopg.readthedocs.org/).
And I would to use *with statement* for transaction handling like:
with (yield from cursor.transaction()): yield from cursor.execute(sql)
The problem is: at exit of *with statement* I need to call `yield from cursor.execute('COMMIT')` or `yield from cursor.execute('ROLLBACK')`.
I can do it only in __exit__ in *context manager*, but python understand only if __exit__: - returns true value, that suppresses exception from code block - returns None or any false value to propagate exception if any - raises exception itself
I propose to add new rule: IF the code object is generator (co_flags & CO_GENERATOR) and __exit__ returns generator object (isinstance(ret, types.GeneratorType)) THEN do `yield from ret`.
That's work fine if __exit__ itself is a *generator function* (contains `yield` or `yield from` statements): call to *generator function* returns *generator object*.
The proposal: 1. Doesn't break any existing code except if user accidentally returns generator object instead of True from __exit__ call (he should not to do this and I sure this is very rare case). 2. Don't requires new syntax.
asyncio itself uses:
with (yield from lock): BLOCK
for locking etc but unlocking for asyncio objects doesn't requires any `yield from`, so __exit__ code is just plain function but not generator.
Also I can live with asyncio trick for __enter__: https://code.google.com/p/tulip/source/browse/asyncio/locks.py#156 The way is a but annoying but unrolling a value returned by __enter__ if the value is generator object will break existing code, sure.
Thoughts?
-- Thanks, Andrew Svetlov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
-- Thanks, Andrew Svetlov
-- --Guido van Rossum (python.org/~guido)
On Sun, Apr 6, 2014 at 6:04 PM, Guido van Rossum <guido@python.org> wrote:
Well, for new syntax, the bar is pretty high... And we should probably move to python-ideas.
And Guido's time machine strikes again! ;) -- Devin
On Sun, Apr 6, 2014 at 4:44 PM, Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
Well, this is good point..
But from my experience try/finally is a common point of errors. Beginners use "with" properly as:
with open(filename) as f: f.read()
but switch to try/finally produces very common error.
Instead of
f = open(filename) try: f.read() finally: f.close()
people usually write
try: f = open(filename) f.read() finally: f.close()
I saw it constantly many times. When I wrote an article about the problem and true solution in my blog I got several emails from my readers: please, tell me again why I need to move open() out of try block if it can generate exception on file opening?
So maybe you would like some other syntax? For example
with from obj: BLOCK
That form can use magic methods with names different than __enter__/__exit__ (and get rid of "with (yield from lock)" BTW). And it's obviously? points on two suspending points: at BLOCK enter and exit.
On Mon, Apr 7, 2014 at 2:12 AM, Guido van Rossum <guido@python.org> wrote:
I prefer to use a try/finally statement. One of the points of using yield-from is that you can always tell where your code may be suspended by searching for "yield from". With your proposed change that would no longer be true -- any with statement would also have to be inspected, and there would no longer be a way to know from the source alone whether it might yield or not (because it would dynamic -- there's no way to be sure at compile time which context manager is being used).
On Sun, Apr 6, 2014 at 12:02 PM, Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
Literally it may be generator itself or function that returns generator object.
Now I'm working on postgres library for asyncio (http://aiopg.readthedocs.org/).
And I would to use *with statement* for transaction handling like:
with (yield from cursor.transaction()): yield from cursor.execute(sql)
The problem is: at exit of *with statement* I need to call `yield from cursor.execute('COMMIT')` or `yield from cursor.execute('ROLLBACK')`.
I can do it only in __exit__ in *context manager*, but python understand only if __exit__: - returns true value, that suppresses exception from code block - returns None or any false value to propagate exception if any - raises exception itself
I propose to add new rule: IF the code object is generator (co_flags & CO_GENERATOR) and __exit__ returns generator object (isinstance(ret, types.GeneratorType)) THEN do `yield from ret`.
That's work fine if __exit__ itself is a *generator function* (contains `yield` or `yield from` statements): call to *generator function* returns *generator object*.
The proposal: 1. Doesn't break any existing code except if user accidentally returns generator object instead of True from __exit__ call (he should not to do this and I sure this is very rare case). 2. Don't requires new syntax.
asyncio itself uses:
with (yield from lock): BLOCK
for locking etc but unlocking for asyncio objects doesn't requires any `yield from`, so __exit__ code is just plain function but not generator.
Also I can live with asyncio trick for __enter__: https://code.google.com/p/tulip/source/browse/asyncio/locks.py#156 The way is a but annoying but unrolling a value returned by __enter__ if the value is generator object will break existing code, sure.
Thoughts?
-- Thanks, Andrew Svetlov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
-- Thanks, Andrew Svetlov
-- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 6 Apr 2014 21:13, "Devin Jeanpierre" <jeanpierreda@gmail.com> wrote:
On Sun, Apr 6, 2014 at 6:04 PM, Guido van Rossum <guido@python.org> wrote:
Well, for new syntax, the bar is pretty high... And we should probably
move
to python-ideas.
And Guido's time machine strikes again! ;)
Also relevant to this topic: http://python-notes.curiousefficiency.org/en/latest/pep_ideas/async_programm... There's a review of what does and doesn't currently work before that last subsection. (Short version: I'm currently +0 on syntax that allows asynchronous transactions and comprehensions, but would need to see some before/after comparisons with real asyncio code to get me to a +1) Cheers, Nick. P.S. I put the time machine keys back where I found them ;)
-- Devin
On Sun, Apr 6, 2014 at 4:44 PM, Andrew Svetlov <andrew.svetlov@gmail.com
wrote:
Well, this is good point..
But from my experience try/finally is a common point of errors. Beginners use "with" properly as:
with open(filename) as f: f.read()
but switch to try/finally produces very common error.
Instead of
f = open(filename) try: f.read() finally: f.close()
people usually write
try: f = open(filename) f.read() finally: f.close()
I saw it constantly many times. When I wrote an article about the problem and true solution in my blog I got several emails from my readers: please, tell me again why I need to move open() out of try block if it can generate exception on file opening?
So maybe you would like some other syntax? For example
with from obj: BLOCK
That form can use magic methods with names different than __enter__/__exit__ (and get rid of "with (yield from lock)" BTW). And it's obviously? points on two suspending points: at BLOCK enter and exit.
On Mon, Apr 7, 2014 at 2:12 AM, Guido van Rossum <guido@python.org>
I prefer to use a try/finally statement. One of the points of using yield-from is that you can always tell where your code may be suspended by searching for "yield from". With your proposed change that would no longer be true -- any with statement would also have to be inspected, and
wrote: there
would no longer be a way to know from the source alone whether it might yield or not (because it would dynamic -- there's no way to be sure at compile time which context manager is being used).
On Sun, Apr 6, 2014 at 12:02 PM, Andrew Svetlov <andrew.svetlov@gmail.com> wrote:
Literally it may be generator itself or function that returns
generator
object.
Now I'm working on postgres library for asyncio (http://aiopg.readthedocs.org/).
And I would to use *with statement* for transaction handling like:
with (yield from cursor.transaction()): yield from cursor.execute(sql)
The problem is: at exit of *with statement* I need to call `yield from cursor.execute('COMMIT')` or `yield from cursor.execute('ROLLBACK')`.
I can do it only in __exit__ in *context manager*, but python understand only if __exit__: - returns true value, that suppresses exception from code block - returns None or any false value to propagate exception if any - raises exception itself
I propose to add new rule: IF the code object is generator (co_flags & CO_GENERATOR) and __exit__ returns generator object (isinstance(ret, types.GeneratorType)) THEN do `yield from ret`.
That's work fine if __exit__ itself is a *generator function* (contains `yield` or `yield from` statements): call to *generator function* returns *generator object*.
The proposal: 1. Doesn't break any existing code except if user accidentally returns generator object instead of True from __exit__ call (he should not to do this and I sure this is very rare case). 2. Don't requires new syntax.
asyncio itself uses:
with (yield from lock): BLOCK
for locking etc but unlocking for asyncio objects doesn't requires any `yield from`, so __exit__ code is just plain function but not generator.
Also I can live with asyncio trick for __enter__: https://code.google.com/p/tulip/source/browse/asyncio/locks.py#156 The way is a but annoying but unrolling a value returned by __enter__ if the value is generator object will break existing code, sure.
Thoughts?
-- Thanks, Andrew Svetlov _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido)
-- Thanks, Andrew Svetlov
-- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
participants (4)
-
Andrew Svetlov -
Devin Jeanpierre -
Guido van Rossum -
Nick Coghlan