[Twisted] Re: posix_spawn would really speed up spawnProcess if we used it

On 14 Mar 2023, at 22:38, Glyph wrote:

Aaaaaand it's landed. Fastest feature on Twisted ever ;-)

How does it perform on linux system? Barry

-g

...

On Mar 14, 2023, at 1:26 PM, Glyph wrote:

Oh hey it looks like… I already did this and it already got reviewed and already got approved and is waiting to get merged: https://github.com/twisted/twisted/pull/1675

but it predated the review robot so it fell through the cracks.

Looks like I'll land this as soon as tests can pass :)

-g

...

On Mar 14, 2023, at 1:15 PM, Glyph wrote:

I am writing some code that was spawning a lot of processes with spawnProcess, and it was going a bit slower than I expected; it was taking about 51 seconds on my M1 Max mac. So I hopped over to my local copy of Twisted and applied this patch:

1 file changed, 9 insertions(+) src/twisted/internet/process.py | 9 +++++++++

modified src/twisted/internet/process.py @@ -367,6 +367,15 @@ class _BaseProcess(BaseProcess): @type environment: L{dict}. @param kwargs: keyword arguments to L{_setupChild} method. """ + from os import posix_spawn, POSIX_SPAWN_DUP2, POSIX_SPAWN_CLOSE + fdmap = kwargs.get("fdmap", {}) + actions = [ + (POSIX_SPAWN_DUP2, childFD, parentFD) + for (parentFD, childFD) in fdmap.items() + ] + self.pid = posix_spawn(executable, args, environment, file_actions=actions) + return + collectorEnabled = gc.isenabled() gc.disable() try:

This is obviously not a complete reimplementation and might have some slight performance advantage due to its incompleteness, but the task in question then took 16 seconds; i.e. a 319% speedup.

Anyone want to do this implementation for real? Or want to commit to a review, if I did it?

-g

_______________________________________________ Twisted mailing list -- twisted@python.org To unsubscribe send an email to twisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/HT7S4IXLH3X... Code of Conduct: https://twisted.org/conduct

_______________________________________________ Twisted mailing list -- twisted@python.org To unsubscribe send an email to twisted-leave@python.org https://mail.python.org/mailman3/lists/twisted.python.org/ Message archived at https://mail.python.org/archives/list/twisted@python.org/message/7JFFYLSN2V2... Code of Conduct: https://twisted.org/conduct