[Python-Dev] PEP 492 quibble and request

[Nick Coghlan]

On 30 April 2015 at 10:33, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> To really understand all implementation details of this line
> you need to read PEP 3156 and experiment with asyncio. There
> is no easier way, unfortunately.  I can't add a super detailed
> explanation how event loops can be implemented in PEP 492,
> that's not in its scope.

This request isn't about understanding the implementation details,
it's about understanding what Python *users* will gain from the PEP
without *needing* to understand the implementation details.

For that, I'd like to see some not-completely-trivial example code in
(or at least linked from) the PEP written using:

* trollius (no "yield from", Python 2 compatible, akin to Twisted's
inlineDeferred's)
* asyncio/tulip ("yield from", Python 3.3+ compatible)
* PEP 492 (async/await, Python 3.5+ only)

The *intent* of PEP 492, like PEP 380 and PEP 342 before it, is "make
asynchronous programming in Python easier". I think it will actually
succeed in that goal, but I don't think it currently does an
especially job of explaining that to folks that aren't both already
deeply invested in the explicitly asynchronous programming model *and*
thoroughly aware of the fact that most of us need asynchronous
programming to look as much like synchronous programming as possible
in order for it to fit our brains. Some folks can fit ravioli code
with callbacks going everywhere in their brains, but I can't, and it's
my experience that most other folks can't either. This lack means the
PEP that gets confused objections from folks that wish explicitly
asynchronous programming models would just go away entirely (they
won't), as well as from folks that already understand it and don't see
why we can't continue treating it as a special case of other features
that folks have to learn how to use from first principles, rather than
saving them that up front learning cost by doing the pattern
extraction to make event driven explicitly asynchronous programming
its own first class concept with dedicated syntax (a hint on that
front: with statements are just particular patterns for using
try/except/finally, decorators are just particular patterns in using
higher order functions, for statements are just particular patterns in
using while statements and builtins, and even imports and classes just
represent particular patterns in combining dictionaries, code
execution and the metaclass machinery - the pattern extraction and
dedicated syntax associated with all of them makes it possible to
learn to *use* these concepts without first having to learn how to
*implement* them)

>From my own perspective, I've spent a reasonable amount of time
attempting to explain to folks the "modal" nature of generators, in
that you can use them both as pure iterable data sources *and* as
coroutines (as per PEP 342).

The problem I've found is that our current approach ends up failing
the "conceptually different things should also look superficially
different" test: outside certain data pipeline processing problems,
generators-as-iterators and generators-as-coroutines mostly end up
being fundamentally *different* ways of approaching a programming
problem, but the current shared syntax encourages users to attempt to
place them in the same mental bucket. Those users will remain
eternally confused until they learn to place them in two different
buckets despite the shared syntax (the 5 or so different meanings of
"static" in C++ come to mind at this point...).

With explicitly asynchronous development*, this problem of needing to
learn to segment the world into two pieces is particularly important,
and this wonderful rant on red & blue functions published a couple of
months ago helps explain why:
http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/

The dedicated async/await syntax proposed in PEP 492 lets the end user
mostly *not care* about all the messiness that needs to happen under
the covers to make explicitly asynchronous programming as close to
normal synchronous programming as possible. The fact that under the
hood in CPython normal functions, coroutines, and generators are
implemented largely as variant behaviours of the same type (producing
respectively the requested result, an object expecting to be driven by
an event loop to produce the requested result, and an object expecting
to be driven be an iterative loop to produce a succession of requested
values rather than a single result) can (and at least arguably should)
be irrelevant to the mental model formed by future Python programmers
(see http://uxoslo.com/2014/01/14/ux-hints-why-mental-models-matter/
for more on the difference between the representational models we
present directly to end users and the underlying implementation models
we use as programmers to actually make things work)

Regards,
Nick.

P.S. *While it's not a substitute for explicitly asynchronous
development, implicitly asynchronous code still has an important role
to play as one of the 3 models (together with thread pools and
blocking on the event loop while running a coroutine to completion)
that lets synchronous code play nice with asynchronous code:
http://python-notes.curiousefficiency.org/en/latest/pep_ideas/async_programming.html#gevent-and-pep-3156

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia