> Your project is fundamentally about instantiating processes?
Yes, that and spawning other subprocesses depending on the outcome of the previous one, sometimes concurrently per-host but mostly sequentially, and clearly most of the time will be spent waiting for suprocesses because all my commands.
> Then any overhead you'd see from threading rather than
> asyncio is going to be completely lost in the noise.
Absolutely, and if I were to use this to orchestrate a bunch of subprocess on thousands of servers, it'd probably happen on a blade with hundreds of gigs of RAM anyway and many cores, so I don't think hardware is really going to be a problem here.
That said, I'm also considering a rewrite of my asyncio code in gevent with stdlib patching, I was already happy with eventlet back when doing OpenStack code, and gevent seems even better, might be the opportunity to get the best of both worlds.
Sorry I think this discussion got out of the scope of this list, I'll be back with more thoughts on python-list in a few weeks after I gather() more insight
Have a great weekend ;)