On Thu, Apr 30, 2015 at 1:36 PM, Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Apr 30, 2015 at 11:48:21AM +0200, Todd wrote:
Looking at pep 492, it seems to me the handling of "for" loops has use outside of just asyncio. The primary use-case I can think of is multiprocessing and multithreading.
For example, you could create a multiprocessing pool, and let the pool handle the items in a "for" loop, like so:
from multiprocessing import Pool
mypool = Pool(10, maxtasksperchild=2)
mypool for item in items: do_something_here do_something_else do_yet_another_thing
A parallel version of map makes sense, because the semantics of map are well defined: given a function f and a sequence [a, b, c, ...] it creates a new sequence [f(a), f(b), f(c), ...]. The assumption is that f is a pure-function which is side-effect free (if it isn't, you're going to have a bad time). The specific order in which a, b, c etc. are processed doesn't matter. If it does matter, then map is the wrong way to process it.
multiprocessing.Pool.map guarantees ordering. It is multiprocessing.Pool.imap_unordered that doesn't.
Of course this sort of thing is possible with iterators and maps today, but I think a lot of the same advantages that apply to asyncio also apply to these sorts of cases. So I think that, rather than having a special keyword just for asyncio, I think it would be better to have a more flexible approach. Perhaps something like a "__for__" magic method that lets a class implement "for" loop handling, along with the corresponding changes in how the language processes the "for" loop.
"async for" hasn't proven itself yet, and you are already looking to generalise it? Shouldn't it prove itself as not a mistake first?
Two reasons: 1. It may be hard to generalize it later without breaking backwards compatibility. 2. Whether it can be generalized or not may have some bearing on whether it gets accepted.