[Twisted-Python] t.p.syslog, t.p.logger and twistd (was #7549)
Moving thread here to avoid polluting trac (thx to glyph). 1. the real issue: t.p.syslog doesn't support logLevel - if you think it's worth a patch before switching to t.p.logger I'll provide a patch (it's quite simple). - if you think we should just move to t.p.logger and get rid of t.p.log and t.p.syslog see 2. 2. moving twistd to t.p.logger - I've started looking around - t.p.logger doesn't still support syslog - I'll try to add some tests - feedback welcome :D Ref: https://twistedmatrix.com/trac/ticket/7549 Peace, R. -- Roberto Polli Community Manager Babel - a business unit of Par-Tec S.p.A. - http://www.babel.it T: +39.06.9826.9651 M: +39.340.652.2736 F: +39.06.9826.9680 P.zza S.Benedetto da Norcia, 33 - 00040 Pomezia (Roma) CONFIDENZIALE: Questo messaggio ed i suoi allegati sono di carattere confidenziale per i destinatari in indirizzo. E' vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale. Se ricevuto per errore, l'uso del contenuto e' proibito; si prega di comunicarlo al mittente e cancellarlo immediatamente.
On Aug 10, 2014, at 3:17 PM, Roberto Polli
Moving thread here to avoid polluting trac (thx to glyph).
1. the real issue: t.p.syslog doesn't support logLevel
For me, the biggest issue is that t.p.syslog defaults to logging at INFO, which means that on many platforms, all the messages get filtered and don't actually get logs. This is particularly true on OS X, where I think filtering works slightly differently and the INFO filtering happens in the process itself, before it even gets sent to the log server. I haven't even figured out how to configure it not to get filtered. It seems like getting t.p.syslog to support logLevel is a necessary part of a clean fix, but since most Twisted log messages don't set a level, we would still need better handling on the commandline to make this practically useful to users.
- if you think it's worth a patch before switching to t.p.logger I'll provide a patch (it's quite simple). - if you think we should just move to t.p.logger and get rid of t.p.log and t.p.syslog see 2.
We definitely can't "get rid of" either of these things. As per the compatibility policy https://twistedmatrix.com/trac/wiki/CompatibilityPolicy we would need to deprecate them. However, "twistd --syslog" could be implemented in terms of t.p.logger instead. Also, since messages are fairly carefully relayed between new and old logging systems, I think it might be a compatible change to make twisted.python.syslog.startLogging use a new-style (twisted.python.logging) observer, rather than fixing SyslogObserver. The benefit here would be that the new logging system consistently uses levels everywhere. On the other hand, any code written to the new API should be populating the old logLevels key via https://github.com/twisted/twisted/blob/bd7f43fa202cb78d23098dee165df58737ff....
2. moving twistd to t.p.logger - I've started looking around - t.p.logger doesn't still support syslog
Yes, we could definitely say that is a blocker :-).
- I'll try to add some tests - feedback welcome :D
I hope some other folks have some ideas :-).
CONFIDENZIALE: …
I'd like to make it clear that messages sent to this list are NOT confidential - they are publicly archived in multiple places :-). If you can disable a message like this when sending here that would be nice. -glyph
On Sunday 10 August 2014 23:09:15 Glyph wrote:
... on OS X, where I think filtering works slightly differently and the INFO filtering happens in the process itself What does tcpdump says on OSX?
However, "twistd --syslog" could be implemented in terms of t.p.logger instead. to make twisted.python.syslog.startLogging use a new-style (twisted.python.logging) observer, rather than fixing SyslogObserver. Ok. Hope to play on that during these holidays.
any code written to the new API should be populating the old logLevels key via <https://github.com/twisted/twisted/blob/bd7f43fa202cb78d23098dee165df58737 ff9192/twisted/python/logger/_legacy.py#L166-L167>. Ok, so: 1- I won't touch twistd 2- I'll create t.p.logger._syslog 3- I''ll patch t.p.syslog to use t.p.logger._syslog
I'd like to make it clear that messages sent to this list are NOT confidential ok, actually the "confidential" just applies when you're not the intended recipient of the mail :P
Thx for your guidance + Peace, R. -- Roberto Polli Community Manager Babel - a business unit of Par-Tec S.p.A. - http://www.babel.it T: +39.06.9826.9651 M: +39.340.652.2736 F: +39.06.9826.9680 P.zza S.Benedetto da Norcia, 33 - 00040 Pomezia (Roma)
On Aug 12, 2014, at 6:24 AM, Roberto Polli
On Sunday 10 August 2014 23:09:15 Glyph wrote:
... on OS X, where I think filtering works slightly differently and the INFO filtering happens in the process itself What does tcpdump says on OSX?'
I am pretty sure that syslog stuff all happens over mach ports, or perhaps UNIX sockets. Nothing that tcpdump can see unless you configure the daemon to forward stuff on.
However, "twistd --syslog" could be implemented in terms of t.p.logger instead. to make twisted.python.syslog.startLogging use a new-style (twisted.python.logging) observer, rather than fixing SyslogObserver. Ok. Hope to play on that during these holidays.
Fantastic :). Keep in mind that as yet, logger is unreleased and so if you have any feedback we should fix it now :).
any code written to the new API should be populating the old logLevels key via <https://github.com/twisted/twisted/blob/bd7f43fa202cb78d23098dee165df58737 ff9192/twisted/python/logger/_legacy.py#L166-L167>. Ok, so: 1- I won't touch twistd 2- I'll create t.p.logger._syslog 3- I''ll patch t.p.syslog to use t.p.logger._syslog
This all sounds great.
Thx for your guidance + Peace, R.
Glad to help. Thanks so much for your contribution! -glyph
Hi @all, On Tuesday 12 August 2014 11:18:31 Glyph Lefkowitz wrote:
2- I'll create t.p.logger._syslog 3- I''ll patch t.p.syslog to use t.p.logger._syslog While writing 2, I found that
1- t.p.logger.LogLevel implements a mapping between syslog priorities and loglevels. 2- there's a private method which actually which doesn't seem to work as expected (priorities order is reversed): * LogLevel._priorityForLevel(LogLevel.debug) == 0 * should be 7 3- To have a correct mapping we should add `kern` even if probably it's not a value which fits to a python application. 4- There's an old closed ticket on that, but it doesn't say anything about loglevels but only an interesting discussion about NamedConstants Q1- Should we implement a priority mapping in LogLevel? Q2- Should we fix that method and make it public? Thx + Peace, R: -- Roberto Polli Community Manager Babel - a business unit of Par-Tec S.p.A. - http://www.babel.it T: +39.06.9826.9651 M: +39.340.652.2736 F: +39.06.9826.9680 P.zza S.Benedetto da Norcia, 33 - 00040 Pomezia (Roma)
On Aug 18, 2014, at 9:14 AM, Roberto Polli
Hi @all,
On Tuesday 12 August 2014 11:18:31 Glyph Lefkowitz wrote:
2- I'll create t.p.logger._syslog 3- I''ll patch t.p.syslog to use t.p.logger._syslog While writing 2, I found that
1- t.p.logger.LogLevel implements a mapping between syslog priorities and loglevels.
Yeah, I remember working on this and I was actually a little surprised we didn't take it further and implement syslog entirely. So I'm glad you're working on this.
2- there's a private method which actually which doesn't seem to work as expected (priorities order is reversed):
Sounds like that should be fixed! Perhaps you could submit a ticket which was just this fix, to get something landed more quickly? :-).
* LogLevel._priorityForLevel(LogLevel.debug) == 0 * should be 7 3- To have a correct mapping we should add `kern` even if probably it's not a value which fits to a python application.
Thoroughness in implementation of specifications is generally a virtue.
4- There's an old closed ticket on that, but it doesn't say anything about loglevels but only an interesting discussion about NamedConstants
Link? If you think that ticket ended up being about something else, you can always file a new one. Thanks for looking it up first though, it's always good to avoid duplicates when we can.
Q1- Should we implement a priority mapping in LogLevel? Q2- Should we fix that method and make it public?
It doesn't have to be public just because it's used by two parts of the log system implementation. Public methods in Twisted are also considered "published"; it's usually better to call private methods internally than to expose a whole bunch of implementation details to application code which it can then start depending upon and making it impossible for us to fix without breaking the compatibility policy. -glyph
participants (3)
-
Glyph -
Glyph Lefkowitz
-
Roberto Polli