[Twisted-Python] Logging and SpawnProcess
![](https://secure.gravatar.com/avatar/125edaeb93ed639010b71693593f47f4.jpg?s=120&d=mm&r=g)
Can anyone point me to some sample code for handling logging with SpawnProcess? Right now I'm using an EchoProtocol that takes stdout and resends it to the log. But it's kludgy because some sub-processes use stdin/stderr for marshaling data. What I'd most like to do is separate data transfer from log output in a clean way.
![](https://secure.gravatar.com/avatar/bcb6ef473ff1644fddee1b4e7c730b01.jpg?s=120&d=mm&r=g)
I'm not entirely following the question (is the log output intermixed with other data on stdout?) but I think that you might find the implementation of twisted.internet.endpoints.ProcessEndpoint a useful example. It relays stderr to the logging stream: https://github.com/twisted/twisted/blob/c8064075a207af1fc9ce19d8f885a986bc5a... ---Tom On Wed, Dec 30, 2020, at 11:38 AM, Robert DiFalco wrote:
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
One minor point, if this is within your power: don't marshal data over stdout; sometimes libraries scribble on it. Open an arbitrary additional file descriptor - say, 7 - and put your marshalled data there. Definitely never use stderr for marshaling anything but lines of text - you're not gonna get away with that :-). Even well-behaved libraries sometimes dump random text to stderr, that's what it's there for, so you don't do it with stdout! Subunit has some really interesting ideas here - it's not Twisted code, but it does deal with communicating structured data over stdio streams in such a way that random overzealous SSH MOTD messages won't ruin your day: https://github.com/testing-cabal/subunit/tree/0e9f67b9683bf2c8c82edb4e511d9b... <https://github.com/testing-cabal/subunit/tree/0e9f67b9683bf2c8c82edb4e511d9b...> Hope this is helpful, -g
![](https://secure.gravatar.com/avatar/bcb6ef473ff1644fddee1b4e7c730b01.jpg?s=120&d=mm&r=g)
I'm not entirely following the question (is the log output intermixed with other data on stdout?) but I think that you might find the implementation of twisted.internet.endpoints.ProcessEndpoint a useful example. It relays stderr to the logging stream: https://github.com/twisted/twisted/blob/c8064075a207af1fc9ce19d8f885a986bc5a... ---Tom On Wed, Dec 30, 2020, at 11:38 AM, Robert DiFalco wrote:
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
One minor point, if this is within your power: don't marshal data over stdout; sometimes libraries scribble on it. Open an arbitrary additional file descriptor - say, 7 - and put your marshalled data there. Definitely never use stderr for marshaling anything but lines of text - you're not gonna get away with that :-). Even well-behaved libraries sometimes dump random text to stderr, that's what it's there for, so you don't do it with stdout! Subunit has some really interesting ideas here - it's not Twisted code, but it does deal with communicating structured data over stdio streams in such a way that random overzealous SSH MOTD messages won't ruin your day: https://github.com/testing-cabal/subunit/tree/0e9f67b9683bf2c8c82edb4e511d9b... <https://github.com/testing-cabal/subunit/tree/0e9f67b9683bf2c8c82edb4e511d9b...> Hope this is helpful, -g
participants (3)
-
Glyph
-
Robert DiFalco
-
Tom Most