[Python-ideas] PEP 525: Asynchronous Generators

[Nick Coghlan]

On 11 August 2016 at 03:17, Sven R. Kunze <srkunze at mail.de> wrote:

> On 09.08.2016 05:23, Nick Coghlan wrote:
>
> On 9 August 2016 at 08:37, Sven R. Kunze <srkunze at mail.de> wrote:
>
>> From what I've heard in the wild, that most if not all pieces of async
>> are mirroring existing Python features. So, building async basically builds
>> a parallel structure in Python resembling Python. Async generators complete
>> the picture. Some people (including me) are concerned by this because they
>> feel that having two "almost the same pieces" is not necessarily a good
>> thing to have. And not necessarily bad but it feels like duplicating code
>> all over the place especially as existing functions are incompatible with
>> async.
>>
>
> It's a known problem that applies to programming language design in
> general rather than being Python specific: http://journal.stuffwithstuff.
> com/2015/02/01/what-color-is-your-function/
>
>
> If it's a such well-known **problem**, why does it even exist in Python in
> the first place? ;-)
>

Because sync vs async *isn't* primarily about parallelism - it's about
different approaches to modelling a problem space. The key differences are
akin to those between functional programming and object-oriented
programming, rather than to those between using threads and processes for
parallel code execution (when all you want is an even lighter weight
alternative to threads without changing the programming model, then you
want something like gevent).

Glyph (of Twisted) wrote a good post about this back when asyncio was first
being considered for the standard library:
https://glyph.twistedmatrix.com/2014/02/unyielding.html

So Python has support for both synchronous and asynchronous programming
constructs for much the same reason we have both closures and classes:
while technically isomorphic capabilities at a theoretical language design
level, in practice, different problems lend themselves to different
modelling techniques.

I don't buy that one couldn't have avoided it.
>

You may want to check out some of the more event-driven programming
oriented languages I mention in
http://www.curiousefficiency.org/posts/2015/10/languages-to-improve-your-python.html#event-driven-programming-javascript-go-erlang-elixir

The easy way to avoid it is to just not offer one of the other, the same
way that pure functional languages don't offer the ability to define
objects with mutable state, and primarily object-oriented languages may not
offer support for full lexical closures.


> Lately, I talked to friend of mine about async and his initial reaction
> was like "hmm that reminds me of early programming days, where you have to
> explicitly tell the scheduler to get control back". He's much older than
> me, so I think it was interesting for him to see that history is repeating
> again.
>

Yep, cooperative multi-threading predated pre-emptive multi-threading by
*many* years (one of the big upgrades between Mac OS 9 and OS X was finally
getting a pre-emptive scheduler). The difference is that where an OS is
scheduling CPU access between mutually unaware applications, application
developers can assume that other control flows *within* the application
space are *not* actively hostile.

In that latter context, enabling local reasoning about where context
switches may happen can be valuable from a maintainability perspective,
hence the pursuit of "best of both worlds" approaches now that the
programming language design community has extensive experience with both.


> One of the big mindset shifts it encourages is to design as many support
> libraries as possible as computational pipelines and message-drive state
> machines, rather than coupling them directly to IO operations (which is the
> way many of them work today). Brett Cannon started the Sans IO information
> project to discuss this concept at http://sans-io.readthedocs.io/
>
>
> Interesting shift indeed and I like small Lego bricks I can use to build a
> big house.
>
> However, couldn't this be achieved without splitting the community?
>

You assume the current confusion around how the sync and async worlds
interrelate will be permanent - it won't, any more than the confusion
around the introduction of lexical closures and new-style classes was
permanent back in the 2.2 era.

It will be a process of convergence, first with the disparate async
communities (Twisted, Tornado, gevent, etc) converging on the async/await
syntactic abstraction on the asynchronous side of things, and then with
standardisation of the sync/async gateways to make async implementations
easy to consume from synchronous code (thread and process pools mean
consumption of synchronous code from async is already relatively
standardised)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160811/6307b08a/attachment-0001.html>