[stdlib-sig] Breaking out the stdlib

[Collin Winter]

On Tue, Sep 15, 2009 at 3:49 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mardi 15 septembre 2009 à 14:38 +0000, Collin Winter a écrit :
>> This is no
>> different than upgrading versions of Postgres, gcc or Linux: it has
>> risks, but gcc doesn't upgrade itself on its own.
>
> But there *is* a difference between unforeseen bugs (which are an
> unavoidable pitfall of software development), and deliberate removal of
> widely used features.
>
> Your reasoning is like saying: there will be unforeseen bugs anyway, so
> we can just create deliberate bugs for the sake of it.

I was specifically responding to the idea that deprecation/removal
creates "ticking time bombs". Willful deprecation is not a bug.

>> Frankly, this sounds like an *excellent* reason to get rid of optparse
>> and replace it with something more flexible.
>
> It *would* have been an excellent reason to choose argparse over
> optparse in the first place. But since optparse is already in the
> stdlib, we have users to care about.
>
>> This is a problem for
>> users of that package, who may wish to use newer version of Python for
>> performance or bug-fix reasons,
>
> This is a reason for keeping that package in the stdlib, not for
> removing it.
>
>> and it is also a problem for
>> python-dev, since those frozen packages create inertia.
>
> I'm not sure what inertia you're talking about. The presence of the
> Lib/optparse.py file does not look like a deterrent to contributing to
> any other part of the interpreter or the stdlib.

Sorry, I meant frozen/unmaintained packages that use optparse, rather
than optparse itself.

> Lib/optparse.py itself
> doesn't seem to receive a lot of checkins, and most are cosmetic. There
> are very few open bugs concerning optparse in bugs.python.org, and most
> seem normal/low priority. From this I gather that:
> 1) optparse is low maintenance (requires few checkins)
> 2) optparse is low annoyance (doesn't clutter the bug tracker)
>
> Taking optparse as an example defeats your point more than it helps it,
> IMO. You should choose a bug-ridden module with an awful API, not
> something perfectly decent which fits lots of everyday use cases.

Did you read what Laura wrote about her experience with optparse?

"""
So -- real use case time -- the hospitals where I have done a lot of
work have some really weird equipment.  And using them costs real
money, uses up real lab supplies, and conceivably can ruin a sample
that you will find it inconvenient or impossible to replace.

There are all sorts of weird dependencies among the various options
you can use to operate the devices.  If you have specified option K,
you may not also specify option M or N, and if you have specified
option L, you must also specify option Q, or option R and option S.

Thus the whole exercise of writing a script to use the equipment
becomes as matter of validating the options you selected are complete
and non-contradictory, and then going out and exercising the hardware.

You can build a validating option parser by hand (option d) or using
getopt (option a).  You will find it difficult to the point of near
impossibility to build one using optparse, because optparse specifically
rejects the notion of 'required options' which is the meat-and-potoatoes
part of this app.  I found this out when optparse went into the standard
library, and was touted at being superior to getopt.

I tried it, and tried to subclass it, and talked to its author about
whether it could be changed to support my use case, and even volunteered
to write code to change it, but was firmly told that optparse worked
the way it did, on purpose, in order to prevent the sort of use that
I wanted to make of it, and that patches to make that possible were
entirely unwelcome.
"""

Did you read what Steven Bethard wrote about his experience with optparse?

"""
On Thu, Sep 10, 2009 at 3:55 PM, Collin Winter <collinw at gmail.com> wrote:
> Honest question, having only read the docs about argparse: would it be
> possible to merge the functionality of argparse into optparse and so
> preserve a greater measure of backwards compatibility? Some of the
> stuff I'm reading about in
> http://argparse.googlecode.com/svn/trunk/doc/argparse-vs-optparse.html
> looks like it should be able to be integrated fairly easily into the
> existing optparse structure.

I tried this, and when I originally started argparse, it was my intent
to make it fully compatible with optparse. For a simple example,
consider the documented "parser.largs", "parser.rargs" and
"parser.values" attributes of OptionParser. Supporting these would not
allow argparse's current parsing strategy, which doesn't follow the
optparse approach of moving args from "largs" to "rargs".
"""

or

"""
On Sun, Sep 13, 2009 at 12:31 AM, Ronald Oussoren
<ronaldoussoren at mac.com> wrote:
> optparse's source code isn't cast in stone, therefore its interface can
> evolve.

While I agree with this comment in general, because optparse
officially exposes lots of internal details (OptionParser.largs,
OptionParser.rargs, Option.TYPE_CHECKER, Option.ALWAYS_TYPED_ACTIONS,
etc.) in practice it's extremely hard to evolve the optparse codebase
without introducing backwards compatibilities. If you don't believe
me, I recommend trying to implement argparse's support for
type=<callable> in optparse, while respecting Option.TYPES, etc. This
is one of the reasons in argparse I've been extremely conservative on
what constitutes the public API - I don't want the compatibility
nightmare that optparse has.
"""

It's also worth noting that optparse continues to have bugs filed
against it, so it is not perfect or low-maintenance in that respect:
http://bugs.python.org/issue2931 is an interesting example.

Collin Winter