survey: is shelve broken? should it be fixed?

[Alex Martelli]

holger krekel wrote:
        ...
>> Say my program uses frameworks A, B, and C.  How ever should I know
>> which of those frameworks uses shelve, and how it uses it?  What if
        ...
> I wonder if this isn't actually only of theoretic importance in our
> special case...

Quite possible: I have no idea which 3rd party Python libraries use
'shelve'.  But as you propose the global-variable-fix as a _general_
approach, the importance grows to surely more than theoretical.

For the specific case of 'shelve', how do you propose to find out?


> Globally that's a valid point. I'd add that different behaviours
> for shelve in a largish application don't make things easier. If you

Note that the shelve _files_ are not changed by this, which does not
make the problem go away but VASTLY lowers it.

> have a shelve instance somewhere you don't really know which behaviour
> it adheres to.

If a shelve.open has an EXPLICIT useoldway=1 or whatever, I do really
know.

With you solution, I don't any more.

Explicit is better than implicit, remember?


> I argued in another posting that with the 'global switch'
> you can encapsulate version-dependent *fixes* basically like so:
> 
>     assert(pythonversion>2.2.4)
>     if pythonversion < 3.0:
>         shelve.returncopies=0
>         somemodule.otherfix=1
> 
> This program will run *without changes* on newer (fixed)
> python-versions. 

...if NO library module it uses (from either the standard Python
library or any 3rd party) is ever coded to depend on the old
behavior of either shelve or somemodule.  Quite a huge *IF*.

> And if python3 or python4 don't want
> to take care of old broken behaviour this script will still run
> fine.

With the same caveat.

 
> Plus if you insist on using the old behaviour (to be
> compatible with python2 and python1.5.2) you can say:

Rather, to freely use library modules whose compatibility
requirements I don't (and can't) really know.

>     assert(pythonversion<3.0)
>     if pythonversion > 2.3:
>         shelve.returncopies=1
> 
> and your code will run on every version up to the one
> where the broken behaviour is not allowed any more.

Except it will break if any library module it uses is
ever upgraded to depend on the NEW behavior.


> Generally i would *try* to avoid scattering version-dependent
> code all over the place. This is really annoying for
> authors and users.

Nothing can be more annoying than global variables, "secretly"
(from the point of view of a 3rd party module) affecting the
behavior of standard library modules the 3rd party module
relies on.  "Version-dependent code" should ideally be in ONE
place ***PER INDEPENDENTLY TESTED AND DISTRIBUTED PACKAGE***,
rather than "all over the place".

But if the choice is, either "all over the place", or in just
one place *PER PROGRAM* (independent of modular structure of
libraries used), then the choice is clear, and "all over the
place" has a huge advantage.


> If this doesn't convince you maybe you could agree to this:
> 
> - have a global default value
> 
> - have a keyword-argument that modifies the instance's behaviour
>   and in the absence of the keyword-argument defaults to the
>   global value.

This is one possibility that can help packages written in the
future, but the keyword argument still won't be used in any
already existing package.  I.e., any reusable production code,
intended to be distributed etc, can hopefully ignore the deuced
global variable and use 'explicit is better than implicit'.

There will have to be an easy way to set the global so that
every lack of an explicit argument raises a warning, of course.
When doing triage with existing packages one will use that,
as well as when testing new packages one is writing or
upgrading.

I don't think this approach would have substantial advantages,
and I can easily think of a zillion traps it could lead users
into, but it's surely far less horrid than just relying on
the global.


> This changes very few lines of your patch and everyone

This is the least of issues -- my tiny patch can well be
regarded as a throw-away proof-of-concept.  The point is
to package up the functionality and its upgrade procedure
in a good way.  Implementing it really well is another
issue (which my path didn't really tackle) but should in
a logical sense come afterwards.

> willing to mix behaviours in large programs could still do
> so. Then i could at least choose  to globally work with the
> 'fixed' behaviour rather than passing keyword-arguments
> everywhere.  Note that a large program mixing behaviour will be
> incompatible with older python versions *and* incompatible
> with new-enough versions (unless python keeps the switch in forever,
> sigh).

If we now encourage people to explicitly ask for either specific 
kind of behavior of course we'll have to keep that forever -- no
asking package authors to put in an explicit switch now and HAVE
to take it out again in a few versions, that's not nice.


> More generally, we are talking about a scheme to introduce *fixes* for
> 'unpythonic/unexpected/astonishing' behaviour that needs
> to be preserved for one or two years in case somebody relied
> on it. But we should *try* to make it easy for someone to choose to
> 
> - rely on old behaviour (drop compatibility with new-enough pythons)
> 
> - rely on new behaviour (drop comptatibility with too-old pythons)
> 
> - rely on mixed behaviours (drop compatibility with too-old and too-new
>   pythons).

I'm not sure the "unsurprising" behavior can be implemented with ZERO
issues so that we only need to preserve the old way "for one or two
years".  The current, surprising behavior may have intrinsic advantages
of performance in some cases, for example.


Alex