[Python-Dev] Using logging in the stdlib and its unit tests

[Vinay Sajip]

Paul Moore <p.f.moore <at> gmail.com> writes:

> And yet Unix invented the concept of stderr, precisely to ensure that
> there's a route for things the program wants to inform the user about
> to get out.
> 
> The NullHandler approach could be seen as the equivalent of adding
> 2>/dev/null to every command by default.
>

Not quite true. The program can (as of right now) ensure that all warnings and
messages get directed to stderr, simply by calling basicConfig(). That'll work
even if every library has added a NullHandler. Although you can argue that the
call to basicConfig() shouldn't be necessary, you tone comes across as a bit of
an overreaction.

> The "specific behaviour" I want is for errors and warnings to go to
> stderr. Every time. (Unless I'm writing a service that wants to
> redirect all output to a log file, or something like that - and that's
> where I'm really glad to have the logging module! And the fact that
> the defaults tend to encourage library authors not to use it really
> frustrate me as an applicatioo developer.)
>

I don't see where your assertion that "the defaults tend to encourage library
developers not to use it" comes from. If it's Antoine's post in this thread,
then I believe he was referring to the stdlib rather than libraries in general,
and I would also say that I don't believe it's a major factor; I would say that
inertia was a more pertinent factor. Of course I understand I could be wrong
about this, but I don't recall when a stdlib maintainer has said to me, "I want
to start using logging in stdlib module XXX, but can't justify it because ..." I
doubt whether making this change will result in a flurry of stdlib modules which
suddenly adopt logging, but stdlib maintainers can certainly prove me wrong
easily enough by stating otherwise in response to this. It's not as if I'm not
approachable - you will see that many improvements to the logging package exist
because of user feedback, both from python-dev and elsewhere.

> Hmm. As far as I can tell from reading the thread, everyone but you is
> in favour of errors and warnings going to stderr by default. I may
> have missed some dissenting views, but it certainly feels like a
> consensus to me!

"Everyone but you" - I think you mean "everyone on this thread but you", and
"everyone else on this thread" is not remotely close to "everyone else, period".
 
> If you feel that the proposal needs to be presented to a wider group
> than just python-dev then that's fair. But personally, I think that
> python-dev will be a reasonably good indicator in this case.

I don't dispute that, but there's not really been enough time for more than a
few people on python-dev to respond.
 
> > But that behaviour is *not* always wanted. That's why I was asked to add
> > warnings integration to logging - to redirect warning messages to logging.
> > Things are seldom as black and white as you're making them out to be here.
> 
> Exactly!!!! Logging should be a BETTER option than direct writes to
> stderr. At the moment, though, it's a CHOICE - writes to stderr, which
> are visible by default but (very) inflexible, or logging which is
> flexible but suppressed by default (OK, "if you follow the recommended
> practice to avoid a user-hostile warning"). The problem is that some
> people are voting with their feet that visibility matters more than
> flexibility.
>

I don't think it's unreasonable to ask which people are voting with their feet,
that you know of. This discussion was initiated by me after I got added to the
nosy list of an issue. It's not as if there has been a host of voters who have
asked for change and are walking away because of some intransigence on my part.
As you can see in my response to Nick's post, I am making concrete suggestions
on how the situation could be improved. It's not helping to talk about people
voting with their feet, especially as no serious attempt to engage with me on
this issue has been made in the past. Because of backward-compatibility
considerations, I would consider it premature to make this kind of change just
based on a request by one or two python-dev members. I have the highest level of
respect for people who have responded on this thread, but how can I be sure that
they're representing everyone else on this list? It's a great community, with a
lot of cohesion, but people do have divergent opinions. (I'm not talking about
me here. If there's a consensus about this, I've no problem implementing a
"handler of last resort".)

> But you don't because the library developer added a NullHandler which
> you have to switch off!!!
>

Where are you getting this "switch off" from? There's no switching off that
happens. The NullHandler remains in place. What you are doing is "switching on"
output to some specific place you want - console, file etc. - with a level of
verbosity and format that you choose.

> FWIW, basicConfig has *never* been what I want. Except in the sense
> that I have to use it to switch on logging to stderr (if only it
> worked, but see the comment above about NullHandler) because that's
> not the default. In all other cases, I've wanted a little bit more
> (which may not be hard to achieve, but that's not the point here).

Well, "may not be hard to achieve" is in line with "not too inconvenient", if
you ask me. And as for "if only it worked", that makes it sound like there's
some effect you can't achieve at very modest cost in terms of any code you have
to write.

> Saying that's convenient is like saying that adding
> sys.stderr=open("/dev/tty", "w") is a convenient way of getting errors
> displayed when you don't like the default destination of /dev/null.

I don't think I'm saying that at all. I see where you're coming from, but apart
from stating your point of view, IMO you're not adding to the discussion in the
same way that Nick, say, has.

Regards,


Vinay Sajip