To answer my own question, it looks like i can just pass sqlalchemy's ResultProxy into deferToThread to make a blocking call to fetch the next set of results: @defer.inlineCallbacks def stream_results(): engine = sqlalchemy.create_engine(...) query = "select * from table" proxy = yield threads.deferToThread(engine.execute, query) while True: results = yield threads.deferToThread(proxy.fetchmany, 1000) if not results: break print results On Thu, Dec 3, 2015 at 2:41 PM, Naveen Michaud-Agrawal < naveen.michaudagrawal@gmail.com> wrote:

I'm trying to use sqlalchemy from a twisted application (by running all blocking queries using deferToThread). Is it possible to yield from within the function running in deferToThread? For example:

def threadRunQuery(engine, query): conn = engine.connect() res = conn.execute(query)

while True: results = res.fetchmany(1000) if not results: break yield results

@defer.inlineCallbacks def stream_results(): engine = sqlalchemy.create_engine(...) query = "select * from table" result_iter = yield threads.deferToThread(threadRunQuery, engine, query) for results in result_iter: print results

It seems that the thread returns a generator, and so everything within threadRunQuery is actually running on the main reactor thread. Is there anyway to stream back results from a deferToThread?

Regards Naveen Michaud-Agrawal

-- ----------------------------------- Naveen Michaud-Agrawal