On Mon, Jun 17, 2013 at 11:50 PM, Glyph <glyph@twistedmatrix.com> wrote:
On Jun 17, 2013, at 5:48 PM, James Y Knight <foom@fuhm.net> wrote:

And having a logging system which doesn't even support a basic log level is just silly. Hopefully the new system can at least have that.
The new system being proposed does have log levels.  (And, for that matter, so does Twisted currently; we've had log levels for compatibility with stlib Python logging forever.)

But it is impossible to use them in Twisted currently without Python logging or without writing your own observer AFAICS. If the system being proposed does have log levels then it is good.

I still don't think that log levels are a particularly useful bit of structured information, and this is one reason I want to have our own structured format, to make sure that the other bits of more useful information hang around for longer in a useful form.  

I think you are mixing "structured information" and "structured format" here. Anyway, current practice proves that log level is one of the most useful pieces of whatever structured or unstructured information in the log files. IMHO applies of course.
I've been convinced that it's unhelpful to be contrarian and omit information which can be useful to a whole bunch of other systems and existing practices.  (Also, the effort described therein is way too ambitious to do in any reasonable time frame unless someone wanted to make logging in Twisted their full-time job for at least a year.)  Plus, I've seen some utility in Calendar Server from the use of the intersection of "level" and "namespace", although blanket application of log levels is still a crapshoot.

That is usually true for any somewhat complex  phenomena - there is no silver bullet, everything should be used in concert. Take gender for example - taken alone it doesn't tell everything about a person but still is a very important piece of information :) Frankly I do not remember a program or a system where log level was not used in one form or another if logging was used at all.

(So, other than those caveats, everything I said about identifying the audience and intent of messages in <http://glyph.twistedmatrix.com/2009/06/who-wants-to-know.html> still applies.)

I agree. It is so tempting to design something that will rule them all :) Unfortunately such a generic project usually is destined not to be completed :( (like Lore to Sphinx conversion ::() (I hope that the number of smiles is enough to indicate the level of seriousness in the above paragraphs ;} ) Seriously, the only things that could probably be general enough to provide ready made interface among gazillions of applications/domains are the time stamp and the log level. The third one - the source of the message is also general but its values are very application specific. Everything else is too application specific to hard code into generic library like Twisted. There should be a way to implement them if needed of course. Among these only log level is missing in Twisted and logPrefix should be fixed to always show correct names.

From the practical POV the most welcomed first step in updating the Twisted log system would be introduction of ready made log levels (with an ability to filter on them in observers), fix of logPrefix thing and somewhat extended set of available observers (syslog, email, socket, etc.) Everything else could wait more detailed design etc.

Do all the systems you mentioned have the same set of log levels, or will there be some need to harmonize them?

IMHO the level of consensus in the set of log levels is not all that bad. The ones used in Python logging together with ability to add custom levels is good enough for almost anything.

Mikhail Terekhov