[Python-Dev] A more flexible task creation

[Steve Holden]

On Thu, Jun 14, 2018 at 8:14 PM, Chris Barker via Python-Dev <
python-dev at python.org> wrote:

> Excuse my ignorance (or maybe it's a vocabulary thing), but I'm trying to
> understand the problem here.
>


> So why do queries fail with 10000 tasks? or ANY number? If the async DB
> access code is written right, a given query should not "await" unless it is
> in a safe state to do so.
>
> So what am I missing here???
>
> because threads ARE concurrent, and there is no advantage to having more
>> threads than can actually run at once, and having many more does cause
>> thread-switching performance issues.
>>
>
> To me, tasks are (somewhat) logically analogous to threads.
>>
>
> kinda -- in the sense that they are run (and completed) in arbitrary
> order, But they are different, and that difference is key to this issue.
>
> As Yury expressed interest in this idea, there must be something I'm
> missing.
>
> What is it?
>

All tasks need resources, and bookkeeping for such tasks is likely to slow
things down. More importantly, with an uncontrolled number of tasks you can
require an uncontrolled use of resources, decreasing efficiency to levels
well below that attainable with sensible conservation of resources.
Imagine, if you will, a task that starts by allocating 1GB of memory. Would
you want 10,000 of those?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180615/1acfed1d/attachment.html>