[stdlib-sig] Evolving the Standard Library

[Michael Foord]

Armin Ronacher wrote:
> Hi everybody,
>
> I'm known for my dislike of the standard libray.  In the past I wrote
> some blog posts about this topic into my personal blog already.  However
> as many people pointed out earlier, a blog is not the place for this
> kind of criticism.  Not only that, also just ranting about a topic does
> not help at all.  Yesterday I subscribed to the stdlib-sig and
> immediately tons of mails ended up in my inbox.  A quick look at the
> mail archives confirms what I was afraid of: this list is really high
> traffic.  I tried to read up some of the discussions I missed but it's
> nearly impossible to do that.
>
>   

And so you repeat a lot of points that have been discussed already in 
the last few days (the only few days in recent years that this list has 
been high traffic).

> [snip...]
>
> Also we have many modules in the standard library that in my opinion
> just do not belong there.  From my point of view, stuff like XML does
> not belong into the standard library.  But it appears that not many
> people agree with me on this one.  

You're right, a lot of people disagree with you. :-)

> But even if everybody would,
> backwards compatibility would still be a good reason to keep these
> modules around.
>
> Besides modules that do not work in every environment or modules that
> were probably a mistake to include, we also have modules in the standard
> library with a hideous implementation or no reusability, forcing people
> to reinvent what's already there.  For a long time, `urllib` was a
> module I would have listed there, but as of Python 2.6, the module
> largely improved by exposing the underlaying socket more which finally
> alllows us to set the timeout in a reliable way.  But there are still a
> ton of modules in the library that cause troubles for people.  `dis` is
> one of them.  The implementation of dis prints to stdout no matter what
> you do.  Of course you can replace sys.stdout with something else for a
> brief moment, but again: this is not something we should aim for or
> advertise because it breaks for many people.  

That particular problem sounds easy to fix.

> `Cookie` is a module
> people monkey patched for a while (badly) to support the http only flag.
>  Not only does the code expose a weird API, it is also nearly impossible
> to extend and even ships cookie subclasses that use unsigned pickles and
> trust the client.  `cgi` has again, shared state on the global namespace
> that alters the behavior of the lirbary.  Of course it was never
> intended to be used by anything but `cgi`, but that leaves people
> reimplementing it or abusing it.
>
>   

cgi and Cookie would both be *excellent* targets for refactoring / 
improving. This is *hugely* preferable to complaining about them. ;-)

I *hope* that Python-dev would be willing to accept some measure of 
backwards incompatibility in the name of improving what few people will 
disagree are potentially useful but horribly outdated APIs.

> So when the discussion started replacing `optparse` with `argparse`,
> because the former is unmaintained I became alerted.  My wishes have
> always been the standard library to be a reliable fallback to be used if
> everything else fails.  Something I can rely on which will not change,
> except for maybe some additions or modules moved to different locations.
> As Python developers we became used to moving import locations a lot.
> It it's `cPickle` or any of the element tree implementations, you name it.
>   

For many things that is an admirable goal. But as you point out, and we 
have already discussed at great length, there is a problem with what to 
do with modules that can't be evolved to meet new requirements because 
of original API design.


> I wonder if the solution to this problem wouldn't be a largely improved
> packaging system and some sort of standardized reviewing process for the
> standard library.  Currently there is not even an accepted style for
> modules ending up in the Python distribution. 

Yes there is - the standard procedure for getting new mdoules into the 
standard library is via the PEP process.

>  That, and a group of
> people, dedicated to standard library refactoring.  The majority of
> libraries in the standard library are small and easy to understand, I'm
> sure they are perfectly suited for students on projects like GSOC or
> GHOP to work on.  They could even be used as some sort of "playground"
> for new Python developers.
>   

It would be a great project for GHOP *if* we have some experienced 
developers, like yourself, dedicated to working out what the things that 
need fixing are.

I think that one of the best ways to achieve the changes you discuss 
below may well be 'one-step-at-a-time' rather than a huge project.

Michael

> Ubuntu recently started the "100 paper cuts" project.  There people work
> on tiny little patches to improve the system, rather to replace
> components.  Even though a large place of the standard library appears
> to be broken by design they could still be redesigned on the small
> scale, without breaking backwards compatibility.
>
> Of course libraries like `locale` and `logging` are hard to change, but
> it would still be possible.  For `locale` it would probably a useful
> idea to go into the direction of datetime, where the timezone
> information is left to a 3rd party library.  `locale` could provide some
> hooks for libraries like `babel` to fill the gap.  On the other hand
> `Cookie` would be very easy to fix by moving the parsing code into a
> separate function and refactoring the cookie objects.
>
> We could probably also start a poll out there with well-selected
> questions of what users think about parts of the library.  And for that
> poll it would make a lot of sense to not just ask the questions and
> evaluating the results, but also track the area the user is coming from
> (small size company, open / closed source, web development etc.).
> Because we all are biased and seeing results grouped by some of these
> factoids could be enlightening.  That said, it could tell us that I'm
> completely wrong with my ideas of how the state of the standard library.
>
> But how realistic is it to refactor the standard library?  I don't know.
> For a long time people were pretty sure Python will not get any faster
> and yet Unleaden Swallow is doing some really amazing progress.
>
> If we want to push Python foward into new areas, and the web is one of
> them, it is necessary to jump into the cold water and start things.
>
> Any maybe we should have some elected task forces for things like the
> standard library.  Judging from the mailinglist it appears that far too
> many people are discussing *every detail* of it.  It is a good idea to
> ask as many people as possible, but I am not sure if the mailinglist is
> the way to do that.  It is currently very hard to see the direction in
> which development is heading.
>
>
> Please think of this email just as a suggestion.  I don't have too much
> trust into myself to follow the discussions on this list camely enough
> to become a real part of a solution, but I would love to help shifting
> the development into a better direction, no matter which one it will be.
>
>
> Regards,
> Armin
> _______________________________________________
> stdlib-sig mailing list
> stdlib-sig at python.org
> http://mail.python.org/mailman/listinfo/stdlib-sig
>   


-- 
http://www.ironpythoninaction.com/