[stdlib-sig] Breaking out the stdlib

[Brett Cannon]

On Tue, Sep 15, 2009 at 07:38, Collin Winter <collinw at gmail.com> wrote:
> On Tue, Sep 15, 2009 at 10:29 AM, Laura Creighton <lac at openend.se> wrote:
[snip]
>> As far as I can tell most software packages go through three stages:
> [snip stage descriptions]
>> Now software in Stage A doesn't need maintenance so much as development
>> in the first place.  And software in stage B requires a lot of
>> maintainance.  But most software in stage C requires little or no
>> maintenance, precisely because it is unchanging.  So, if you decide to
>> change how Exceptions are inherited, for instance, you may break a
>> whole lot of Stage C: code, but fixing them is part of the general
>> problem of 'fixing exceptions everywhere' not part of 'fixing getopt'.
>>
>> Now I think that there is some confusion here between packages who
>> are in Stage C:  and packages who are in Stage B-2:  around here.
>> The B-2 packages are probably the core developers greatest headache.
>> But I don't see the C packages as being troublesome at all.  If you
>> don't like them, it isn't because of the work that maintaining is
>> costing you.  It may be that hatred for the B-2's has become a general
>> hatred of all packages with no maintainers, which is an understandable
>> mistake.  But from reading this list I get the distinct impression that
>> some people just hate C:s _precisely because they are old and unchanging_,
>> and would continue to hate them for that reason even if I was in some
>> way able to guarantee that they would never need any maintenance ever
>> again.  These people are condemning the packages I love best for the
>> reason they are the packages I love best.  And that is the attitude
>> I would like to change.
>
> Speaking as a core developer, I disagree. The problem with
> unmaintained (in your terms, B-2) or intentionally-frozen (C-*)
> packages is that they make it difficult for us to evolve and adapt
> Python the language and Python the standard library: if no-one is
> willing/available to update the code to account for language/library
> changes, the frozen package will become pinned to a specific
> known-good version (or range of versions) of Python. Over time, that
> version of Python will become uncommon (as distros phase it out) and
> unsupported (as python-dev end-of-lifes it). This is a problem for
> users of that package, who may wish to use newer version of Python for
> performance or bug-fix reasons, and it is also a problem for
> python-dev, since those frozen packages create inertia.
>
[snip]
>> It boils down to a matter of trust.  My customers trust me to not give
>> them ticking time bombs that will all stop working one day, and I trust
>> you not to go about gratuitously removing perfectly working code that
>> is quietly sitting there, not needing any changes, and not bothering
>> anybody.  When you break that contract with me, my customers suffer,
>> I suffer, and the people who said 'You shouldn't have coded it in
>> Python in the first place, but picked a mature language like Java'
>> are completely vindicated.
>
> The systems you have written for your customers are not autonomous
> agents; presumably, you have not written code like "if today.year ==
> 2011: sudo apt-get upgrade python" into these systems that would
> change the version of Python running without anyone asking. Human
> beings control these upgrades. If the human being performs the upgrade
> blindly, without taking appropriate risk mitigation steps, there's
> very little that we can do to protect them. You left out a key element
> in the chain of trust above: presumably, you trust your customers not
> to violate the minimum requirements you've set out for the software
> you've written for them. If you say "this software requires 2GB of
> RAM" and the customer later decides to try running the machines with
> only 256MB to save money instead, that's not your fault: it's theirs.
> Likewise, if you say "this software runs on Python 2.5" and they
> blindly install Python 3.1 instead, that's not your fault: it's
> theirs.
>
> As to the matter of Java's deprecation policy, I don't regard it as
> "mature": I regard it as a sign of different requirements. Because you
> can't know what browser version or JRE version a user's desktop is
> running, stability is paramount for Java; "write once, run anywhere"
> is not free, it has its costs. As Frank Wierzbicki has said (either in
> this thread or the other one about argparse), the inability to ever
> remove code from the libraries makes life difficult for Java's
> developers -- who have to maintain this code -- as well as for
> everyday Java engineers, who have to learn to navigate this maze of
> deprecated vs non-deprecated solutions to the same problem. Java shops
> generally end up with a list of Approved Java Classes so that new
> hires and old pros alike don't get tripped up.
>
> In Python, we don't have the luxury of a paid staff to work on our
> libraries, to maintain the crufty, fragile,
> we'd-like-to-get-rid-of-you-but-maybe-someone's-using-you-we-don't-really-know
> modules. We rely almost exclusively on volunteer contributions, and
> it's tough to find volunteers to work on crap code. It's one thing to
> choose not to change something; it's another thing entirely not to be
> able to change something, to have your hands tied by code you can't
> see and no-one will change. As Python development slows, as stability
> gets confused for permanence and stasis, I predict it will be harder
> to attract enthusiastic, eager contributors. After all, who wants to
> work on something you're not allowed to modify?
>
> To speak more personally, and specifically to the issue of
> getopt/optparse vs argparse: at Google, I'm part of the Python
> readability team, which helps train the large numbers of Python
> developers that the company produces. Part of this job involves
> conducting detailed code reviews for new Python programmers,
> explaining both Google style and idiomatic Python code generally,
> suggesting library A over hand-written solution B. I am, frankly,
> embarrassed whenever I have to explain the difference between getopt
> and optparse, urllib and urllib2, popen2 vs os.popen* vs subprocess,
> string.* vs str.*, etc. I cannot imagine how embarrassed I will be
> when I have to explain why the standard library includes getopt,
> optparse and argparse.

I agree with Collin in that we have to keep perspective on how this
impacts active core contributors and potential future ones. Like all
core developers I only have a small amount of free time in my life to
spend working on Python. That means I have to prioritize what patches
I review, what bugs I try to fix, etc., because frankly there are too
many for me and all of the active contributors to look at (I could say
how that is part of why I think the stdlib needs to slim down a
little, but that can wait for a PEP proposing what to trim and why).
So if I am browsing the issue tracker trying to help out, I can come
across getopt and optparse issues. Now I can obviously work on those
and fix bugs, but I also happen to know that argparse is out there,
generally well received by the community, and has been requested for
addition to the stdlib for years (the crowd was rather enthusiastic
during Steven's lightning talk at PyCon 2010 about argparse). It's a
little deflating to work on code that I know the community has deemed
mediocre compared to something out in the wild. Why should I spend my
time on getopt and optparse when I could try to get argparse into the
stdlib and spend my precious free time on some code I know will
ultimately serve the community better than the current code? Or how
about code that slipped in years ago and has not held up well in terms
of design? That's even worse because of the amount of time I have to
burn trying to untangle some coding mess that we have inherited from
Python's history.

I don't want to feel like I am wasting my time when I work on Python
stuff. And I am sure the other core developers don't either. And this
is partially why we have modules that end up with a laundry list of
bug reports and patches languishing in the issue tracker. Nor do I
want Python's reputation tarnished, as Collin pointed out, by having
old modules sitting there in the standard library.

Which gives another reason why argparse is attractive; Steven is
already a core developer and will keep argparse maintained, letting me
slowly ignore getopt/optparse issues guilt-free while I work on other
issues that actually bring me some form of joy.

-Brett