[Twisted-Python] Cause epollreactor to busyspin
![](https://secure.gravatar.com/avatar/20316199a8197e9c349a945df8afcd90.jpg?s=120&d=mm&r=g)
Hi All, I have a latency sensitive application that gets affected when it is moved between cores or the system has to wake from idling. I would like to run the thread in as tight a loop as possible and to do this i need to cause epollreactor to busyspin on receiving data. I have worked out how to do it by editing the twisted code directly by hard coding the value of `timeout` to be 0 at this location in the code https://github.com/twisted/twisted/blob/f074ba3d5083aa1503abcf194aece327e7f8... Is it possible to actually set this on the reactor in a more sensible way so I don't need to patch the twisted codebase. ? Below is basically what I am doing. ``` def doPoll(self, timeout): """ Poll the poller for new events. """ + timeout = 0 - if timeout is None: - timeout = -1 # Wait indefinitely. try: ``` Cheers Tim Hughes
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
Can you explain "gets affected" in more detail? If moving between cores is the issue, could you set its CPU affinity?
I would like to run the thread in as tight a loop as possible and to do this i need to cause epollreactor to busyspin on receiving data.
I am very curious about your application now :)
I have worked out how to do it by editing the twisted code directly by hard coding the value of `timeout` to be 0 at this location in the code https://github.com/twisted/twisted/blob/f074ba3d5083aa1503abcf194aece327e7f8...
Is it possible to actually set this on the reactor in a more sensible way so I don't need to patch the twisted codebase. ? Below is basically what I am doing.
You can achieve this with a tiny bit of extra overhead by doing something like this: def cant_sleep_clown_will_eat_me(): reactor.callLater(0, cant_sleep_clown_will_eat_me) cant_sleep_clown_will_eat_me() Does this actually improve your latency? -glyph
![](https://secure.gravatar.com/avatar/20316199a8197e9c349a945df8afcd90.jpg?s=120&d=mm&r=g)
On 2016-04-21 03:57, Glyph wrote:
That is exactly what I would do with a physical machine but unfortunately where I need to use it is in china and I only have xen virtual machines available. I know this isn't the best situation but unfortunately that is all I can get in these locations.
We are using it to test the performance of streaming financial market data over different CDN providers to different cities around the world. Currently trying to jump through the hoops to be able to opensource it and stick it on github. It isn't that exciting, it is basically a modified LineReceiver with the delimiter set to \x01 and the messages are key=value pairs at a peak rate of approx 130000 pairs per second https://en.wikipedia.org/wiki/Financial_Information_eXchange
Will give this a go
![](https://secure.gravatar.com/avatar/20316199a8197e9c349a945df8afcd90.jpg?s=120&d=mm&r=g)
On 2016-04-21 11:30, Tim Hughes wrote:
Hoops have been jumped and it is on github at https://github.com/LMAX-Exchange/txfixclient
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
Can you explain "gets affected" in more detail? If moving between cores is the issue, could you set its CPU affinity?
I would like to run the thread in as tight a loop as possible and to do this i need to cause epollreactor to busyspin on receiving data.
I am very curious about your application now :)
I have worked out how to do it by editing the twisted code directly by hard coding the value of `timeout` to be 0 at this location in the code https://github.com/twisted/twisted/blob/f074ba3d5083aa1503abcf194aece327e7f8...
Is it possible to actually set this on the reactor in a more sensible way so I don't need to patch the twisted codebase. ? Below is basically what I am doing.
You can achieve this with a tiny bit of extra overhead by doing something like this: def cant_sleep_clown_will_eat_me(): reactor.callLater(0, cant_sleep_clown_will_eat_me) cant_sleep_clown_will_eat_me() Does this actually improve your latency? -glyph
![](https://secure.gravatar.com/avatar/20316199a8197e9c349a945df8afcd90.jpg?s=120&d=mm&r=g)
On 2016-04-21 03:57, Glyph wrote:
That is exactly what I would do with a physical machine but unfortunately where I need to use it is in china and I only have xen virtual machines available. I know this isn't the best situation but unfortunately that is all I can get in these locations.
We are using it to test the performance of streaming financial market data over different CDN providers to different cities around the world. Currently trying to jump through the hoops to be able to opensource it and stick it on github. It isn't that exciting, it is basically a modified LineReceiver with the delimiter set to \x01 and the messages are key=value pairs at a peak rate of approx 130000 pairs per second https://en.wikipedia.org/wiki/Financial_Information_eXchange
Will give this a go
![](https://secure.gravatar.com/avatar/20316199a8197e9c349a945df8afcd90.jpg?s=120&d=mm&r=g)
On 2016-04-21 11:30, Tim Hughes wrote:
Hoops have been jumped and it is on github at https://github.com/LMAX-Exchange/txfixclient
participants (2)
-
Glyph
-
Tim Hughes