[Python-Dev] PEP282 and the warnings framework

[Vinay Sajip]

[Vinay]
> > ...
> > So far, I am not sure what benefit switching from the status quo of
> > integer-based levels to class-based levels will give. Certainly, the
idea of
> > relative levels of importance of events must be preserved, since
> > severity-based filtering is a key function of logging.
 [Holger]
> first off, the DebugMessage/InfoMessage/... could
> have a class-level severity property (integer) if one
> really wants to have it for compatibility etc.

Yes, I am aware of this - as I said in my last post, "...if you use some
level attribute in message classes, what has really been gained by using
classes?" What I mean is, that an integer level is simpler than a class with
an integer level attribute.

> this leaves the question how a Message-instance/type is more
> powerful than a tuple(string/integer).
>
> Some thoughts regarding Message-instances/types:
>
> - the convenient logging API and categorization features
>   need *not* be changed largely (but the signature of the
>   log method should/might be changed)
>
> - beeing forced to totally order all types of messages can
>   be difficult. You may want e.g. to log 'TimeoutMessages'
>   no matter what the current integer threshold might be.
>   why not something like: loginstance.accept(TimeoutMessage)

If timeouts are of concern in a design, you could just log to a channel
called "timeout" or a channel such as "timeout.tcp.connect". If it's agreed
that the idea of a hierarchical namespace of loggers is a good idea (it's
one of log4j's Big Ideas), then it can be used for categorizing log events
either according to functional areas of the application, or implementation
layers of the application, or both. You get finer grained control than with
classes, unless you want to create *lots* of classes.

> - Message instances can defer the *construction* of a string
>   until it is really needed. (see my other posting for a code
>   example). This is more effective if you choose to ignore
>   certain messages. This has *nothing* to do with formatting.

"Construction of a string"? If you are not talking about formatting, then I
presume you mean a string literal or value bound to a string.  If efficiency
is the driver here, then I would have thought initialization of a Python
class instance would be more expensive than a string.

> - customized messages are powerful and allow more advanced
>   filtering than with strings, especially for large apps.
>   a connection-logger could choose to ignore a
>   TimeoutMessage(socket) based on the target address of the
>   socket. doing this with strings is hard (and perlish:-)

Nobody is saying you have to "just use strings". For example, you could use
a customized Filter for this. To take the timeout example further, it might
be that other context is important when trying to log the message - e.g. not
just the socket address, but perhaps also the number of timeouts seen
recently. (After all, the odd timeout is expected - otherwise one wouldn't
design for it.) It might be that such context is more readily available in a
higher-level class (such as an HTTPServer). By using the Filter interface
(any class implementing the filter() method can be a filter), you get more
flexibility even in the scenario you mentioned. The decision is left up to
the application designer, not imposed by the logging system.

> - a SocketMessage-class may hold a weakref-list of sockets allowing
>   to log 'sock<1>', 'sock<2>' instead of addresses which are
>   hard to read.  stuff like this is *very* convenient. Again
>   there is hardly any performance penalty if SocketMessages
>   are ignored.

A Filter class can do this just as effectively. If messages are ignored
because of level configuration, the filter wouldn't even get called, so no
performance penalty would apply.

> Maybe the categorization features of the logging
> API make up for some of the (relative) inflexibilities of
> string/integer messages.

> But what do you really gain by restricting a message
> to a (string,integer) taking my arguments into account?

I'm not saying that a message must be restricted to only strings and
integers. I'm only saying that the existing API provides the flexibility you
need, and more besides. I also feel that the proposal you're endorsing holds
some disbenefits. For example, you have not really responded to my arguments
about interoperability, pickling, or the ability to change levels
dynamically if the developer wants to for a particular application. And now
we're talking about syntactic sugar, rather than using exception classes as
a dispatch mechanism.

Design choices are personal things. For many APIs we could say "but I'd
rather do it like this...". Walter's original post seemed to be motivated by
musing on the similarities between the two frameworks, rather than by a
specific problem with PEP282/logging.py which needed to be addressed. I
don't want to seem negative, or hidebound; a look at logging.py's change
history shows that many changes have been added in response to user
feedback. Much of this feedback has come from people actually using the
system to solve their problems. But I really feel that all of the
flexibility is available already. Kevin Butler's post showed how you could
easily get what Holger wanted from the existing API:

[Kevin]
> trace = logging.getLogger( "trace" )
> trace.debug( "%s", TraceStateMessage( obj ))
[Holger]
>Having to write '%s' in a logging call is not convenient IMO.
...
>nice :-) But does this beat
>    somelog.traceobj(obj)
[Holger]
>Having to write '%s' in a logging call is not convenient IMO.
...

Now, it's getting to be a matter of personal taste rather than some
fundamental problem with functionality. Some might prefer to say

    log.debug(EnvMessage('HOME','points to non-existent directory'))

but I prefer

    log.debug("HOME is a non-existent directory: %s", the_home_value)

in most cases. In the cases where I *do* want more flexibility, the existing
API allows me to do this without *forcing* me to use classes. For example,
in 0.4.5 (not yet released) I've made a minor change in Formatter.format(),
replacing record.msg with str(record.msg). This allows code like this:
#-code--------------------------------------
import logging

class MySpecialClass:
    def __init__(self, param):
        self.param = param

    def __str__(self):
        return "%s, %%s" % self.param

class MyOtherSpecialClass(MySpecialClass):
    def __str__(self):
        return "%s" % self.param

logging.basicConfig()
root = logging.getLogger("")

root.warn(MySpecialClass("Hello"), "world!")
root.warn(MyOtherSpecialClass("Goodbye!"))
#-output-------------------------------------
WARN:root:Hello, world!
WARN:root:Goodbye!
#--------------------------------------------

'Nuff said?

[Holger, replying to Kevin's post]
>People seem to apply their (logging) experience from the
>java/c++ world to python not realizing how much easier it
>is with python to work with objects. Were we to speak
>in java terms i wouldn't bother to argue :-)

Er...not really, at least in this case. Java style seems to proliferate
classes beyond necessity; the use of the Level class in log4j is, to me, an
example of this. If you want an example of a "calqué" (literal) translation
of log4j to Python, see the log4p project on Sourceforge (unmaintained since
2000). I didn't want logging.py to look like log4j, beyond using their best
ideas; it doesn't feel Java-like to me, nor does it have any particular C++
idioms.


Regards


Vinay Sajip



_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com