[Python-Dev] A more flexible task creation

Steve Holden steve at holdenweb.com
Fri Jun 15 03:56:49 EDT 2018

On Thu, Jun 14, 2018 at 8:14 PM, Chris Barker via Python-Dev <
python-dev at python.org> wrote:

> Excuse my ignorance (or maybe it's a vocabulary thing), but I'm trying to
> understand the problem here.

> So why do queries fail with 10000 tasks? or ANY number? If the async DB
> access code is written right, a given query should not "await" unless it is
> in a safe state to do so.
> So what am I missing here???
> because threads ARE concurrent, and there is no advantage to having more
>> threads than can actually run at once, and having many more does cause
>> thread-switching performance issues.
> To me, tasks are (somewhat) logically analogous to threads.
> kinda -- in the sense that they are run (and completed) in arbitrary
> order, But they are different, and that difference is key to this issue.
> As Yury expressed interest in this idea, there must be something I'm
> missing.
> What is it?

All tasks need resources, and bookkeeping for such tasks is likely to slow
things down. More importantly, with an uncontrolled number of tasks you can
require an uncontrolled use of resources, decreasing efficiency to levels
well below that attainable with sensible conservation of resources.
Imagine, if you will, a task that starts by allocating 1GB of memory. Would
you want 10,000 of those?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180615/1acfed1d/attachment.html>

More information about the Python-Dev mailing list