PEP 3156 - Asynchronous IO Support Rebooted
Hello. I've read the PEP and some things raise questions in my consciousness. Here they are. 1. Series of sock_ methods can be organized into a wrapper around sock object. This wrappers can then be saved and used later in async-aware code. This way code like: sock = socket(...) # later, e.g. in connect() yield from tulip.get_event_loop().sock_connect(sock, ...) # later, e.g. in read() data = yield from tulip.get_event_loop().sock_recv(sock, ...) will look like: sock = socket(...) async_sock = tulip.get_event_loop().wrap_socket(sock) # later, e.g. in connect() yield from async_sock.connect(...) # later, e.g. in read() data = yield from async_sock.recv(...) Interface looks cleaner while plain calls (if they ever needed) will be only 5 chars longer. 2. Not as great, but still possible to wrap fd in similar way to make interface simpler. Instead of: add_reader(fd, callback, *args) remove_reader(fd) We can do: wrap_fd(fd).reader = functools.partial(callback, *args) wrap_fd(fd).reader = None # or del wrap_fd(fd).reader 3. Why not use properties (or fields) instead of methods for cancelled, running and done in Future class? I think, it'll be easier to use since I expect such attributes to be accessed as properties. I see it as some javaism since in Java Future have getters for this fields but they are prefixed with 'is'. 4. Why separate exception() from result() for Future class? It does the same as result() but with different interface (return instead of raise). Doesn't this violate the rule "There should be one obvious way to do it"? 5. I think, protocol and transport methods' names are not easy or understanding enough: - write_eof() does not write anything but closes smth, should be close_writing or smth alike; - the same way eof_received() should become smth like receive_closed; - pause() and resume() work with reading only, so they should be suffixed (prefixed) with read(ing), like pause_reading(), resume_reading(). Kind regards, Yuriy.
On Tue, Jan 8, 2013 at 6:53 PM, Benjamin Peterson <benjamin@python.org> wrote:
No, actually, in that case it *does* raise an exception, because it means that the caller didn't understand the interface. It *returns* an exception object when the Future is done but the "result" is exceptional. But it *raises* when the Future is not done yet. -- --Guido van Rossum (python.org/~guido)
On Wed, Jan 9, 2013 at 11:14 AM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
The exception() method exists for the same reason that we support both "key in mapping" and raising KeyError from "mapping[key]": sometimes you want "Look Before You Leap", other times you want to let the exception fly. If you want the latter, just call .result() directly, if you want the former, check .exception() first. Regardless, the Future API isn't really being defined in PEP 3156, as it is mostly inheritied from the previously implemented PEP 3148 (http://www.python.org/dev/peps/pep-3148/#future-objects) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, Jan 8, 2013 at 5:14 PM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
I've read the PEP and some things raise questions in my consciousness. Here they are.
Thanks!
This is a semi-internal API that is mostly useful to Transport implementers, and there won't be many of those. So I prefer the API that has the fewest classes.
Ditto.
Too late, this is how PEP 3148 defined it. It was indeed inspired by Java Futures. However I would defend using methods here, since these are not all that cheap -- they have to acquire and release a lock.
Because it is quite awkward to check for an exception if you have to catch it (4 lines instead of 1).
I am indeed struggling a bit with these names, but "writing an EOF" is actually how I think of this (maybe I am dating myself to the time of mag tapes though :-).
- pause() and resume() work with reading only, so they should be suffixed (prefixed) with read(ing), like pause_reading(), resume_reading().
Agreed. -- --Guido van Rossum (python.org/~guido)
On Tue, Jan 8, 2013 at 8:31 PM, Guido van Rossum <guido@python.org> wrote:
I think I want to take that back. I think it is more common for a protocol to want to pause the transport (i.e. hold back data_received() calls) than it is for a transport to want to pause the protocol (i.e. hold back write() calls). So the more common method can have a shorter name. Also, pause_reading() is almost confusing, since the protocol's method is named data_received(), not read_data(). Also, there's no reason for the protocol to want to pause the *write* (send) actions of the transport -- if wanted to write less it should not have called write(). The reason to distinguish between the two modes of pausing is because it is sometimes useful to "stack" multiple protocols, and then a protocol in the middle of the stack acts as a transport to the protocol next to it (and vice versa). See the discussion on this list previously, e.g. http://mail.python.org/pipermail/python-ideas/2013-January/018522.html (search for the keyword "stack" in this long message to find the relevant section). -- --Guido van Rossum (python.org/~guido)
On Wed, Jan 9, 2013 at 8:50 AM, Guido van Rossum <guido@python.org> wrote:
I totally agree with protocol/transport stacking, anyone should be able to do some ugly thing like FTP over SSL over SOCKS over SSL over HTTP (j/k). Just take a look at what you can do with netgraph in *BSD (anything over anything with any number of layers). But still we shouldn't sacrifice ease of understanding (both docs and code) for couple extra chars (10 actually). Yes, 'reading' is misleading, pause_receiving and resume_receiving are better. -- Kind regards, Yuriy.
On Wed, Jan 9, 2013 at 8:31 AM, Guido van Rossum <guido@python.org> wrote:
Ok, I see. Should transports be bound to event loop on creation? I wonder, what would happen if someone changes current event loop between these calls.
I understand why it should be a method, but still if it's a getter, it should have either get_ or is_ prefix. Are there any way to change this with 'Final' PEP? they do. I've just imagined the amount of words I'll have to say to students about EOFs instead of simple "it closes our end of one half of a socket".
-- Kind regards, Yuriy.
On Tue, Jan 8, 2013 at 9:02 PM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
Yes, this is what the transport implementation does.
Why? That's not a universal coding standard. The names seem clear enough to me.
Are there any way to change this with 'Final' PEP?
No, the concurrent.futures package has been released (I forget if it was Python 3.2 or 3.3) and we're bound to backwards compatibility. Also I really don't think it's a big deal at all.
But which half? A socket is two independent streams, one in each direction. Twisted uses half_close() for this concept but unless you already know what this is for you are left wondering which half. Which is why I like using 'write' in the name. -- --Guido van Rossum (python.org/~guido)
On Wed, Jan 9, 2013 at 9:14 AM, Guido van Rossum <guido@python.org> wrote:
But in theory every sock_ call is independent and returns Future bound to current event loop. So if one change event loop with active transport, nothing bad should happen. Or I'm missing something.
When I see (in autocompletion, for example) or remember name like "running", it triggers thought that it's a field. When I remember smth like is_running, it definitely associates with method.
Yes, not a big deal.
Yes, 'write' part is good, I should mention it. I meant to say that I won't need to explain that there were days when we had to handle a special marker at the end of file. -- Kind regards, Yuriy.
Is this thread really ready to migrate to python-dev when we're still bikeshedding method names? Yuriy Taraday writes:
Mystery is good for students.<wink/> Getting serious, "close_writer" occured to me as a possibility.
Well, if we're at the "bikeshedding about names" stage, that means that no serious issues with the proposal are left. So it's a sign of progress. On Wed, Jan 9, 2013 at 12:42 AM, Stephen J. Turnbull <stephen@xemacs.org>wrote:
-- Jasper
On Tue, Jan 8, 2013 at 9:26 PM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
It is bound to the event loop whose sock_<call>() method you called.
So if one change event loop with active transport, nothing bad should happen. Or I'm missing something.
Changing event loops in the middle of event processing is not a common (or even useful) pattern. You start the event loop and then leave it alone.
That must pretty specific to your personal experience.
But even today you have to mark the end somehow, to distinguish it from "not done yet, more could be coming". The equivalent is typing ^D into a UNIX terminal (or ^Z on Windows). -- --Guido van Rossum (python.org/~guido)
On Wed, Jan 9, 2013 at 10:02 AM, Guido van Rossum <guido@python.org> wrote:
Yes. It was not-so-great morning idea.
My interns told me that they remember EOF as special object only from high school when they had to study Pascal. I guess, in 5 years students won't understand how one can write an EOF. (and schools will finally replace Pascal with Python) -- Kind regards, Yuriy.
On Wed, Jan 9, 2013 at 8:55 PM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
Python really doesn't try to avoid the concept of an End-of-file marker. ================ $ python3 Python 3.2.3 (default, Jun 8 2012, 05:36:09) [GCC 4.7.0 20120507 (Red Hat 4.7.0-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
Only makes one system call, so less data may be returned than requested In non-blocking mode, returns None if no data is available. On end-of-file, returns ''. ================ Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi Yuriy, For the record, it isn't necessary to cross-post. python-ideas is the place for discussing this, and most interested people will be subscribed to both python-ideas and python-dev, and therefore they get duplicate messages. Regards Antoine. Le Wed, 9 Jan 2013 05:14:02 +0400, Yuriy Taraday <yorik.sar@gmail.com> a écrit :
On Tue, Jan 8, 2013 at 6:53 PM, Benjamin Peterson <benjamin@python.org> wrote:
No, actually, in that case it *does* raise an exception, because it means that the caller didn't understand the interface. It *returns* an exception object when the Future is done but the "result" is exceptional. But it *raises* when the Future is not done yet. -- --Guido van Rossum (python.org/~guido)
On Wed, Jan 9, 2013 at 11:14 AM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
The exception() method exists for the same reason that we support both "key in mapping" and raising KeyError from "mapping[key]": sometimes you want "Look Before You Leap", other times you want to let the exception fly. If you want the latter, just call .result() directly, if you want the former, check .exception() first. Regardless, the Future API isn't really being defined in PEP 3156, as it is mostly inheritied from the previously implemented PEP 3148 (http://www.python.org/dev/peps/pep-3148/#future-objects) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, Jan 8, 2013 at 5:14 PM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
I've read the PEP and some things raise questions in my consciousness. Here they are.
Thanks!
This is a semi-internal API that is mostly useful to Transport implementers, and there won't be many of those. So I prefer the API that has the fewest classes.
Ditto.
Too late, this is how PEP 3148 defined it. It was indeed inspired by Java Futures. However I would defend using methods here, since these are not all that cheap -- they have to acquire and release a lock.
Because it is quite awkward to check for an exception if you have to catch it (4 lines instead of 1).
I am indeed struggling a bit with these names, but "writing an EOF" is actually how I think of this (maybe I am dating myself to the time of mag tapes though :-).
- pause() and resume() work with reading only, so they should be suffixed (prefixed) with read(ing), like pause_reading(), resume_reading().
Agreed. -- --Guido van Rossum (python.org/~guido)
On Tue, Jan 8, 2013 at 8:31 PM, Guido van Rossum <guido@python.org> wrote:
I think I want to take that back. I think it is more common for a protocol to want to pause the transport (i.e. hold back data_received() calls) than it is for a transport to want to pause the protocol (i.e. hold back write() calls). So the more common method can have a shorter name. Also, pause_reading() is almost confusing, since the protocol's method is named data_received(), not read_data(). Also, there's no reason for the protocol to want to pause the *write* (send) actions of the transport -- if wanted to write less it should not have called write(). The reason to distinguish between the two modes of pausing is because it is sometimes useful to "stack" multiple protocols, and then a protocol in the middle of the stack acts as a transport to the protocol next to it (and vice versa). See the discussion on this list previously, e.g. http://mail.python.org/pipermail/python-ideas/2013-January/018522.html (search for the keyword "stack" in this long message to find the relevant section). -- --Guido van Rossum (python.org/~guido)
On Wed, Jan 9, 2013 at 8:50 AM, Guido van Rossum <guido@python.org> wrote:
I totally agree with protocol/transport stacking, anyone should be able to do some ugly thing like FTP over SSL over SOCKS over SSL over HTTP (j/k). Just take a look at what you can do with netgraph in *BSD (anything over anything with any number of layers). But still we shouldn't sacrifice ease of understanding (both docs and code) for couple extra chars (10 actually). Yes, 'reading' is misleading, pause_receiving and resume_receiving are better. -- Kind regards, Yuriy.
On Wed, Jan 9, 2013 at 8:31 AM, Guido van Rossum <guido@python.org> wrote:
Ok, I see. Should transports be bound to event loop on creation? I wonder, what would happen if someone changes current event loop between these calls.
I understand why it should be a method, but still if it's a getter, it should have either get_ or is_ prefix. Are there any way to change this with 'Final' PEP? they do. I've just imagined the amount of words I'll have to say to students about EOFs instead of simple "it closes our end of one half of a socket".
-- Kind regards, Yuriy.
On Tue, Jan 8, 2013 at 9:02 PM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
Yes, this is what the transport implementation does.
Why? That's not a universal coding standard. The names seem clear enough to me.
Are there any way to change this with 'Final' PEP?
No, the concurrent.futures package has been released (I forget if it was Python 3.2 or 3.3) and we're bound to backwards compatibility. Also I really don't think it's a big deal at all.
But which half? A socket is two independent streams, one in each direction. Twisted uses half_close() for this concept but unless you already know what this is for you are left wondering which half. Which is why I like using 'write' in the name. -- --Guido van Rossum (python.org/~guido)
On Wed, Jan 9, 2013 at 9:14 AM, Guido van Rossum <guido@python.org> wrote:
But in theory every sock_ call is independent and returns Future bound to current event loop. So if one change event loop with active transport, nothing bad should happen. Or I'm missing something.
When I see (in autocompletion, for example) or remember name like "running", it triggers thought that it's a field. When I remember smth like is_running, it definitely associates with method.
Yes, not a big deal.
Yes, 'write' part is good, I should mention it. I meant to say that I won't need to explain that there were days when we had to handle a special marker at the end of file. -- Kind regards, Yuriy.
Is this thread really ready to migrate to python-dev when we're still bikeshedding method names? Yuriy Taraday writes:
Mystery is good for students.<wink/> Getting serious, "close_writer" occured to me as a possibility.
Well, if we're at the "bikeshedding about names" stage, that means that no serious issues with the proposal are left. So it's a sign of progress. On Wed, Jan 9, 2013 at 12:42 AM, Stephen J. Turnbull <stephen@xemacs.org>wrote:
-- Jasper
On Tue, Jan 8, 2013 at 9:26 PM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
It is bound to the event loop whose sock_<call>() method you called.
So if one change event loop with active transport, nothing bad should happen. Or I'm missing something.
Changing event loops in the middle of event processing is not a common (or even useful) pattern. You start the event loop and then leave it alone.
That must pretty specific to your personal experience.
But even today you have to mark the end somehow, to distinguish it from "not done yet, more could be coming". The equivalent is typing ^D into a UNIX terminal (or ^Z on Windows). -- --Guido van Rossum (python.org/~guido)
On Wed, Jan 9, 2013 at 10:02 AM, Guido van Rossum <guido@python.org> wrote:
Yes. It was not-so-great morning idea.
My interns told me that they remember EOF as special object only from high school when they had to study Pascal. I guess, in 5 years students won't understand how one can write an EOF. (and schools will finally replace Pascal with Python) -- Kind regards, Yuriy.
On Wed, Jan 9, 2013 at 8:55 PM, Yuriy Taraday <yorik.sar@gmail.com> wrote:
Python really doesn't try to avoid the concept of an End-of-file marker. ================ $ python3 Python 3.2.3 (default, Jun 8 2012, 05:36:09) [GCC 4.7.0 20120507 (Red Hat 4.7.0-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
Only makes one system call, so less data may be returned than requested In non-blocking mode, returns None if no data is available. On end-of-file, returns ''. ================ Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Hi Yuriy, For the record, it isn't necessary to cross-post. python-ideas is the place for discussing this, and most interested people will be subscribed to both python-ideas and python-dev, and therefore they get duplicate messages. Regards Antoine. Le Wed, 9 Jan 2013 05:14:02 +0400, Yuriy Taraday <yorik.sar@gmail.com> a écrit :
participants (9)
-
Antoine Pitrou -
Benjamin Peterson -
Dustin J. Mitchell -
Glyph -
Guido van Rossum -
Jasper St. Pierre -
Nick Coghlan -
Stephen J. Turnbull -
Yuriy Taraday