(Somehow your post came in twice. I'm replying to the second one.)This seems a reasonable idea. A problem may be how to specify this, since all positional and keyword arguments to `submit()` after the function object are passed to the call. A possible solution would be to add a second call, `submit_blocking()`, that has the semantics you desire.I recommend that you open an issue on bugs.python.org to discuss the form this API should take, explaining your use case, and in the meantime prepare a PR for the GitHub cpython project to be linked to the issue.Good luck improving Python!--GuidoOn Wed, Sep 4, 2019 at 4:24 AM Chris Simmons <chris.simmons.0@hotmail.com> wrote:I have a script that uploads files to Google Drive. It presently performs the uploads serially, but I want to do the uploads in parallel--with a reasonable number of simultaneous uploads--and see if that improves performance. I think that an Executor is the best way to accomplish this task.
The trouble is that there's no reason for my script to continue queuing uploads while all of the Executor's workers are busy. In theory, if the number of files to upload is large enough, trying to queue all of them could cause the process to run out of memory. Even if it didn't run out of memory, it could consume an excessive amount of memory.
It might be a good idea to add a "block" option to Executor.submit (https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.submit) that allows the caller to block until a worker is free to handle the request. I believe that this would be trivial to implement by passing the "block" option through to the Executor's internal Queue.put call (https://github.com/python/cpython/blob/242c26f53edb965e9808dd918089e664c0223407/Lib/concurrent/futures/thread.py#L176).
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-leave@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VNAGCVY7L33VARFDZJZTGAAQ3FZTQ5GO/
Code of Conduct: http://python.org/psf/codeofconduct/
--_______________________________________________--Guido van Rossum (python.org/~guido)Pronouns: he/him/his (why is my pronoun here?)
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-leave@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/J4PP3FFJOXU5MSASPAN4ZMY6OILR4C3N/
Code of Conduct: http://python.org/psf/codeofconduct/