[Python-Dev] API design: where to add async variants of existing stdlib APIs?

Brett Cannon brett at python.org
Tue Mar 7 12:41:05 EST 2017


I don't think a common practice has bubbled up yet for when there's both
synchronous and asynchronous versions of an API (closest I have seen is
appending an "a" to the async version but that just looks like a spelling
mistake to me most of the time). This is why the question of whether
separate modules are a better idea is coming up.

On Tue, 7 Mar 2017 at 02:24 Michel Desmoulin <desmoulinmichel at gmail.com>
wrote:

> Last week I had to download a CSV from an FTP and push any update on it
> using websocket so asyncio was a natural fit and the network part went
> well.
>
> The surprise was that the CSV part would not work as expected. Usually I
> read csv doing:
>
> import csv
>
> file_like_object = csv_crawler.get_file()
> for row in csv.DictReader(file_like_object)
>
> But it didn't work because file_like_object.read() was a coroutine which
> the csv module doesn't handle.
>
> So I had to do:
>
> import csv
> import io
>
>         raw_bytes = await stream.read(10000000)
>         wrapped_bytes = io.BytesIO(raw_bytes)
>         text = io.TextIOWrapper(wrapped_bytes, encoding=encoding,
> errors='replace')
>
>         for i, row in enumerate(csv.DictReader(text)):
>
> Turns out I used asyncio a bit, and I now the stdlib, the io AIP, etc.
> But for somebody that doesn't, it's not very easy to figure out. Plus
> it's not as elegant as traditional Python. Not to mention it loads the
> entire CSV in memory.
>
> So I wondered if I could fix the csv module so it accept async. But the
> question arised. Where should I put it ?
>
> - Create AsyncDictReader and AsyncReader ?
> - Add inspect.iscoroutine calls widh it in the regular Readers and some
> __aiter__ and __aenter__ ?
> - add a csv.async namespace ?
>
> What API design are we recommanding for expose both sync and async
> behaviors ?
>
>
> Le 07/03/2017 à 03:08, Guido van Rossum a écrit :
> > On Mon, Mar 6, 2017 at 5:57 PM, Raymond Hettinger
> > <raymond.hettinger at gmail.com <mailto:raymond.hettinger at gmail.com>>
> wrote:
> >
> >     Of course, it makes sense that anything not specific to asyncio
> >     should go outside of asyncio.
> >
> >     What I'm more concerned about is what the other places actually
> >     are.   Rather than putting async variants of everything sprinkled
> >     all over the standard library, I suggest collecting them all
> >     together, perhaps in a new asynctools module.
> >
> >
> > That's a tough design choice. I think neither extreme is particularly
> > attractive -- having everything in an asynctools package might also
> > bundle together thing that are entirely unrelated. In the extreme it
> > would be like proposing that all metaclasses should go in a new
> > "metaclasstools" package. I think we did a reasonable job with ABCs:
> > core support goes in abc.py, support for collections ABCs goes into the
> > collections package (in a submodule), and other packages and modules
> > sometimes define ABCs for their own users.
> >
> > Also, in some cases I expect we'll have to create a whole new module
> > instead of updating some ancient piece of code with newfangled async
> > variants to its outdated APIs.
> >
> > --
> > --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> >
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/desmoulinmichel%40gmail.com
> >
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170307/aced14ce/attachment.html>


More information about the Python-Dev mailing list