[Twisted-Python] deferreds and unpacking binary data
Question: A Protocol should unpack some binary data. this is done in seperate calls to struct.unpack . Inbetween i do some checks for length, some strings returned are concatenated using join and logs are written. This all happens in the factory of the Protocol. To keep the code unblocking would it be wiser to spawn new Threads from the Protocol around the whole function, like: d = threads.deferToThread(self.factory.readPackage, data) because it all has to happen after one another anyway. Or should i split it up in deferreds which if i get it correctly would mean to create callback functions for each of the unpack()'s, join()'s and str()'s thanks
On May 10, 2010, at 1:01 PM, Xtroce wrote:
Question: A Protocol should unpack some binary data. this is done in seperate calls to struct.unpack . Inbetween i do some checks for length, some strings returned are concatenated using join and logs are written. This all happens in the factory of the Protocol. To keep the code unblocking would it be wiser to spawn new Threads from the Protocol around the whole function, like: d = threads.deferToThread(self.factory.readPackage, data) because it all has to happen after one another anyway. Or should i split it up in deferreds which if i get it correctly would mean to create callback functions for each of the unpack()'s, join()'s and str()'s thanks
If the work is all synchronous (i.e. in-memory manipulations of data, no I/O) then just do it synchronously in the main thread. If this blocks for a while and you want to parallelize it, it would be a good idea to split it into a separate process rather than doing any of the things you suggest.
participants (2)
-
Glyph Lefkowitz -
Xtroce