[stdlib-sig] Evolving the Standard Library

[Armin Ronacher]

Hi,

Vinay Sajip schrieb:
> It's easy for you to just say "it's broken by design", and that's only a more
> polite way of saying "it sucks". It doesn'nt strike me as a basis for
> constructive dialogue, unless you provide some more specifics.
Of course it is easier.

> I think you are being dogmatic, rather than pragmatic. The Zen of Python says,
> "practicality beats purity." Your bugbear here seems to be how shared state
> causes problems in web applications. Despite having shared state, AFAIK the
> logging module is quite usable in a web context - as well as the usage with
> Django and Tornado that I mentioned earlier, Google App Engine uses it too
> (meaning, all the web applications developed with GAE can use it).
Django has a global settings module and yet there are tons of developers
hating that one.  Even Simon Willison agrees that the settings module
was the biggest mistake made in Django.

> You're a very smart guy, Armin, but you perhaps need to consider that
> it is possible to not like a design because it doesn't suit your personal taste
> - but that doesn't necessarily make it a bad design. 
That does not have anything to do with personal taste or not.  Some
things cannot work in some situations, and logging is currently one of
them.  Logging was designed to be based on persistent loggers.  You get
one, it's there, you can log into it.  SQLAlchemy for example does an
incredible dance to get separate loggers for temporary database connections.

> Currently, Python logging doesn't do formatting of stack traces etc. until it's
> sure that the message is severe enough to require handling, based on the current
> logger configuration.
That is actually something where I had a small problem with the logging
implementation that could be fixed: the exc_text is currently only set
in the format() function of the formatter, instead of being an attribute
on the log record.  And yes, it could be an attribute of the log record
if it was a property.  (I guess the reason it isn't currently is that
the logging module is backwards compatible down to 1.5.2 or something)

> So at present, Python logging conforms to your statement "Only if there
> was any handler that wants to do anything with that message, it would
> pull the details form the stack and format ..."
My point was that there are no loggers, no registry of loggers, just
senders and senders are arbitrary Python objects.

> In Python logging, you never have to instantiate a Formatter in your code unless
> you want some specific formatting functionality or format. In your scheme, if a
> user wanted certain messages to be formatted in certain ways (and other messages
> in other ways - e.g. for a log file as opposed to console display), you would do
> this without any formatter classes - how, exactly?
In my personal experience the formatting is based on the handler.  For
example if I want to log into a text file, i have one formatting, if I
log to stderr I have a different one.  If I log into an email I have a
completely different one.  For example I tried a long ago to log my
exceptions from a logging handler into a new ticket in the trac.  I
ended up replacing a log of code from both log record and formatter
inside the log handler because of the way the message was assembled.  I
had to escape text and could not rely on string formatting to work.

> The global registry of loggers is there to avoid the need to pass loggers around
> the system - you just access them by name. It's a bit like thread locals - do
> you have a problem with thread locals, too?
I have a problem with unnecessarily used thread locals.  Not with the
concept in general.

> Sure, it's state shared across threads, unlike thread locals. But if
> it's causing you a specific problem because you've found some
> non-thread-safe behaviour, I'd really like to know.
I don't complain that it's not threadsafe, I'm pretty sure if logging
was thread unsafe someone would have noticed by now.

> Agreed, the logging from a library leading to warning messages was an annoyance
> in the library as first released. But *you* never raised an issue about it.
That is true and I have to change that.

> If default configuration for the logging system is the biggest reason for you to
> hate that package, don't get mad - get even. Tell me how I messed up and how to
> make it better.
If I would know how to improve it so that I'm happy with it, I would
have told you.

> Slow? Sure, everything has a cost. It's all about tradeoffs. What specific
> performance problems have you come up against? What mitigating strategies did
> you use? What were the observed performance metrics, as against what you
> expected? Did you do any profiling to be sure that logging was definitely the
> culprit? Show me the numbers.


> If your frustration with shared state is from an aesthetic point of view, I
> can't really help. After all, sys.modules (say) is shared state too.
http://lucumr.pocoo.org/2009/7/24/singletons-and-their-problems-in-python
*cough*

> You cannot delete loggers because in a multithreaded application, other threads
> may still be using them. You can, however, disable loggers - which, from a
> functional point of view, seems as good.
Which is why I would not design a logging library based on loggers.

> Or are you finding that loggers are taking up too much memory?
Even if a logger would be just 8 bytes in size, it would steal leak if
you cannot control the number of loggers created.  (see SQLAlchemy for a
nice example).

> That's a cheap remark, I would have expected better from you. So everybody who
> uses logging (and is smart, like you) hates it but uses it because they have no
> choice, right? It *is* widely used, AFAIK. If everyone shared your opinion, I
> don't think that would be the case.
A while ago when I was blogging about logging I wrote this:

> [...] It's called "logging" and does exactly that --- it logs errors.
> I don't know why so many people miss it or just don't use it, but it's
> really one of the good things in the python standard library.

to which Robert Brewer replied:

> That's also why it's so bad: it's so extensible and configurable
> that's it's far too slow for high-performance websites.

And I'm pretty sure Rober knows what he's talking about.

> You can't contribute constructive criticism because you don't know how? Well,
> you've already made some criticisms in your post, so how about putting some in
> more detail? I really like Jinja2 (a lot), but if someone said to you "Jinja2
> sucks, it's really slow, it does all that conversion to bytecode stuff, that's
> really gross. I can't really say any more than that, excuse me while I vomit"
> then how would you feel? That's pretty much how you're coming across right now.
I gave up on defending Jinja2 a while ago.  Because people from the
Django world constantly call me names for enabling logic in templates ;)

I'm sorry for the way I expressed my disagreement with the library's
design here and in what way.  To make you feel better: from all the
modules in the standard library, logging is still one of the best
designed and implemented, despite my disagreement with it.

> Perhaps you think I'm being oldskool by mentioning "courtesy", but I prefer to
> think if it as an attribute of being a grown-up. Open source is very political,
> but that doesn't mean it doesn't have to be polite.
I agree.

> Look at Graham Dumpleton's frustration over WSGI
> and Python 3.0 - I feel more than a little sympathy for his situation. He's
> actually trying to create something useful, just as I am. There's no shortage of
> approaches, and opinions, and in general, I believe competition is good. But
> some of us are just trying to get some work done here.
and WSGI for Python 3 is something that has to be discussed.  Graham's
master plan is one proposal, but in my opinion not the best one.


Regards,
Armi