concurrent.futures vs gevent
jspicklemire at gmail.com
jspicklemire at gmail.com
Mon Aug 24 12:08:51 EDT 2015
Some people have tried to make a case that concurrent.futures should be adopted as a replacement wherever greenlet based algorithms are in use. However, my experience is that greenelts combined with concurrent.futures provides significant advantages. In other words, to a degree the two approaches complement each other.
Specifically, 'greening' a ThreadPoolExecutor delivers the simplicity of either a context manager, or a map, depending on the use case, hadling in a few lines of simple Python what might have otherwise been a complex call-back or 'twisted' solution.
Please note, this usage applies only to ThreadPool, not ProcessPool.
for example:
import eventlet
eventlet.patcher.monkey_patch(os=False, socket=True, select=True, thread=True)
futures = eventlet.import_patched('concurrent.futures') # 'greening' futures,
fut_tp_exec = futures.ThreadPoolExecutor
# the lines above enable even nested map / context manager 'futures'
# adding the lines below enables a vast number of 'simultaneous' HTTP requests.
import requests
req_f_sess = eventlet.import_patched('requests_futures.sessions') # 'green' req
o_rest_sess = req_f_sess.FuturesSession(executor=fut_tp_exec(max_workers=5000))
More information about the Python-list
mailing list