On Thu, Mar 15, 2012 at 3:24 PM, Niki Spahiev wrote:

On 14.03.2012 21:28, anatoly techtonik wrote:

...

Badly need `logging2` module that has NullHandler assigned by default for all loggers. http://packages.python.org/Logbook/api/handlers.html#logbook.NullHandler

Why? Because logging fails to play well with libraries:

   import logging    log = logging.getLogger(__name__)    log.warn("WARN")

   No handlers could be found for logger "spyderlib.utils.bsdsocket"

What do I want from library logging as a Python application developer? Nothing until I explicitly setup default behaviour.

logging can not be changed, and leaving everything as-is is a PITA, that's why I am proposing for logging2 as the only viable solution.

Can't this be solved with new function? e.g.  log = logging.getLibLogger(__name__)

I don't know - there are questions. The function is used to get logger with NullHandler() if application did not provide configuration for root logger. So.. 1. How will logging know that application provided root logger configuration? 2. How will logging know that it should use root logger handler instead of more specific handler for __name__ module? This comes to the second part - what if logging.getLibLogger(__name__) is called after the application root handler is configured? 1. Will a new, more specific NullHandler() override the active root configuration? There is one more concern. If anything in the code base (including 3rd party modules) calls logging.log() or friends before root logger is configured by application, the root logger becomes automatically configured making everything above useless. -- anatoly t.

On Fri, Mar 16, 2012 at 12:32 AM, Cameron Simpson wrote:

On 14Mar2012 21:28, anatoly techtonik wrote: | Badly need `logging2` module that has NullHandler assigned by default | for all loggers. | http://packages.python.org/Logbook/api/handlers.html#logbook.NullHandler | | Why? Because logging fails to play well with libraries: | |    import logging |    log = logging.getLogger(__name__) |    log.warn("WARN") | |    No handlers could be found for logger "spyderlib.utils.bsdsocket" | | What do I want from library logging as a Python application developer? | Nothing until I explicitly setup default behaviour.

Fair point.

Conversely, almost every app I write commences thus:

 from cs.logutils import setup_logging

 def main(argv):    setup_logging()    ... main code ...

That sends to stderr with frills. Finer grained setup can come later.

But that makes all such libraries dependent on cs.logutils, which is not an option. In my case these libraries are self-contained PySide widgets that can be used standalone or from an IDE. Adding dependency on cs.logutils makes them too tied to application.

I _think_ I prefer logging's current behaviour:

 - I do want a big fat warning if I forget to configure logging at all

Unless your application uses alternative logging means and doesn't use logging at all (unlike 3rd party libraries it uses).

 - I don't want lobraries doing sufficient work at import time to    warrant logging anything

I do not want libraries doing any logging setup work at import either. They should just need to contain 'extension points' for plugging log handlers in case I need to debug these modules. -- anatoly t.

On Fri, Mar 16, 2012 at 5:10 AM, Cameron Simpson wrote:

On 16Mar2012 00:39, anatoly techtonik wrote: | > | What do I want from library logging as a Python application developer? | > | Nothing until I explicitly setup default behaviour. | > | > Fair point. | > | > Conversely, almost every app I write commences thus: | >  from cs.logutils import setup_logging | >  def main(argv): | >    setup_logging() | >    ... main code ... | > | > That sends to stderr with frills. Finer grained setup can come later. | | But that makes all such libraries dependent on cs.logutils, which is | not an option.

No, it doesn't. setup_logging just sets up the root logger somewhat. Your apps need some kind of convenience routine like it, is all. It can be very small.

All the libraries have unchanged vanilla logging calls. For example, I use SQLalchemy et al under this scheme. The libraries don't know about cs.logutils at all.

| In my case these libraries are self-contained PySide | widgets that can be used standalone or from an IDE. Adding dependency | on cs.logutils makes them too tied to application.

The libraries don't use my convenience module. My module just makes a whole bunch of setup I happen to like reduce to just two lines of code in my app, import and setup call.

So, your scenario is clear. You have an application where you configure logging and none of your libraries are used without this configuration. This is a doable scenario for logging module. But logging2 is needed for the opposite user story - when you have a bunch of independent libraries that are used from scripts, from application or standalone, and you need to make them silent by default. Doing this requires inserting a logging configuration into the header of every library. Things are dim, because 3rd party libraries don't come with these headers. Moreover - it is unclear if this configuration won't break the logging when you finally have to turn it on for this particular lib from an application.

| > I _think_ I prefer logging's current behaviour: | >  - I do want a big fat warning if I forget to configure logging at all | | Unless your application uses alternative logging means and doesn't use | logging at all (unlike 3rd party libraries it uses).

If the 3rd party libraries use the logging module and you don't want logging happening, just configure logging to throw stuff away.

The problem that I don't know and probably don't want to think about logging schemes used by those libs. When the time comes to troubleshoot these libs then sure - the logging will be the first thing I'll look into.

| >  - I don't want lobraries doing sufficient work at import time to | >    warrant logging anything | | I do not want libraries doing any logging setup work at import either. | They should just need to contain 'extension points' for plugging log | handlers in case I need to debug these modules.

Well, they do. Via logging. Supply an adapter handler to hook their calls to logging to your alternative logging means. Logging doesn't have to log to files/syslog/smtp etc - you can sent it anywhere. I'm sure you know that, so I think I'm missing some subtlety in your use case.

Yes, the argument in not about API for 'extension points' (logging does that). The argument that logging does implicit extra work, and it requires even more extra work to cancel the produced effects. More over, the effects of this default implicit work are non-deterministic - that means they depend on execution flow in you program. The effects will be different if you toss these events in random order: - root config is set by application - logging.log() or friends called - getLogger('some.dotted.name').warn() called Basically, proper logging setup also requires to care about import order of you application. It's not a problem if you have a single entry point, but becomes a PITA with multiple entrypoints or in exploratory programming (IPython?) sessions. "Don't let me think" is the greatest usability principle of all times, and logging module, unfortunately fails to comply. Backward compatibility won't let it to be fixed. And that's why a predictable logging2 module is needed. -- anatoly t.

On Fri, Mar 16, 2012 at 11:53 AM, Paul Moore wrote:

On 16 March 2012 08:33, anatoly techtonik wrote:

...

"Don't let me think" is the greatest usability principle of all times, and logging module, unfortunately fails to comply. Backward compatibility won't let it to be fixed. And that's why a predictable logging2 module is needed.

Your original example was wrong:

The example is actually right, and the preceding string explains that example code should be run from the library.

...

...

...

import logging log = logging.getLogger(__name__) log.warn("WARN") WARNING:__main__:WARN log = logging.getLogger('a.b.c') log.warn("WARN") WARNING:a.b.c:WARN

Restart the interpreter to clear the state and execute only the last part:

...

...

import logging log = logging.getLogger('a.b.c') log.warn("WARN") No handlers could be found for logger "a.b.c"

Now without restarting execute the following:

...

...

log = logging.getLogger('spyderlib.utils.bsdsocket') log.warn("WARN")

Does the original example look correct now? I guess, Paul, that you've overlooked the reference to library, so I skipped the rest of the quote, because you should agree that library should not set logging level for itself as it will cancel logging config may have already occurred at application level. -- anatoly t.

On 16 March 2012 16:03, anatoly techtonik wrote:

I may assume that we are all agree that using logging from libraries with scenarios I've described in Python 2.x is a PITA and all my arguments are valid against this version.

Nope, I checked 2.7 too. I just didn't paste both cases. You should at a minimum, state the version you are using when you make these comments. And anyway, Python 2.x behaviour isn't really an appropriate topic for python-ideas, as there won't be a new 2.x version. Paul.

On Sun, Mar 18, 2012 at 10:54 AM, Stephen J. Turnbull wrote:

On Sat, Mar 17, 2012 at 1:03 AM, anatoly techtonik wrote:

...

problem required a lot of effort alone. If there was a list for known misbehaviours like this in Python 2.x the things could be much easier.

If you don't find it on the tracker (including closed issues), for the purpose of actually getting it changed, it's unknown.

The problem that it is almost impossible to find anything on tracker related to specific component like logging. The issue description doesn't always correspond to the actual content and it is really hard to get through all the comments. Proposal - add modules field, allow comment rating, issue and commend edits (Trac). Why won't I do anything? Because my time is limited to 15 minutes slots and getting developer instance takes more time, so I never pass through this barrier (and TAL is not my language).

...

I guess the question can be closed, but I am still dissatisfied with the current state of things.

So fix it.  If you didn't find an open issue on the tracker, that means either few people (== 0) care enough to post an issue at all, or, as far as those who care are concerned, it's already fixed in the relevant branch(es) and the issue has been closed.  Either way, you're going to have to do it yourself.

...

If Python 3.x is developed using the same process, there is a risk of UX problems like this too.

There's always a risk of problems.  Do you have a specific proposal for improving process?

The first step is to gather a critical mass of people who acknowledge the problem, who have their own ideas summarized and can share them at the right moment. So, the specific proposal is to have a history that can be analyzed by anyone. Right now it is extremely hard to summarize problems - no custom tags on tracker, no way to organize email threads - these are just two technical proposals to improve the process. One more idea is the use case DB to collect API uses cases and detect conflicts at early stage to draw attention to them. Every problem and use case should have a number, should be clearly defined (much better than tracker issue summaries) and should have a rating/star system. That should be enough for now. -- anatoly t.