[Python-Dev] Re: [Distutils] Questions about distutils strategy

[Tim Peters]

Briefly backtracking to an old thread:

[Guido]
> ...
> The problem lies in which key is used.  All versions of
> Python 1.5.x (1.5, 1.5.1, 1.5.2) use the same key!  This
> is a main cause of trouble, because it means that different
> versions cannot peacefully live together even if the user
> installs them into different directories -- they will all
> use the registry keys of the last version installed.  This,
> in turn, means that someone who writes a Python application
> that has a dependency on a particular Python version (and
> which application worth distributing doesn't :-) cannot
> trust that if a Python installation is present, it is the
> right one.  But they also cannot simply bundle the standard
> installer for the correct Python version with their program,
> because its installation would overwrite an existing Python
> application, thus breaking some *other* Python apps that
> the user might already have installed.

Right, that's one class of intractable problem under Windows.

*Inside* my workplace, another kind of problem is caused when people try to
make a Python app available over the Windows network.  They stick the Python
they want and its libraries out on the network, with python.exe in the same
directory as the app.  Now some people have highly customized Python setups,
and the network Python picks up "the wrong" site.py etc.  That sucks, and
there appears no sane way to stop it.

Telling internal app distributors they need to invent a unique registry key
and fiddle their python.exe's resources is a non-starter.  Ditto telling
people with highly customized Pythons "don't do that".  Ditto telling anyone
they have to run any sort of installation script just to use a network app
(sometimes they don't even know they're running it! e.g., when it's a
subsystem invoked by another app).

So while everyone is thinking about the hardest possible scenarios, please
give a thought to the dirt simple one too <0.5 wink>:  an app distributor
who knows exactly what they're doing, and for whom *any* magical inference
is simply a barrier to overcome.  The latter can be satisfied by any number
of means, from an envar that says "please don't try to be helpful, *this* is
the directory you look in, and if you don't find stuff there give up" to a
cmdline switch that says the same.  Nothing Windows-specific there -- any OS
with an envar or a cmdline will play along <wink>.

> ...
> I thought a bit about how VB solves this.  I think that when
> you wrap up a VB app in, all the support code (mostly a big
> DLL) is wrapped with it.  When the user runs the installer,
> the DLL is installed (probably in the WINDOWS directory).  If
> a user installs several VB apps built with the same VB
> version, they all attempt to install the exact same DLL; of
> course the installers notice this and optimize it away, keeping
> a reference count.

This is the way most *MS* DLLs work; stuff like the C runtime libraries and
MS database drivers work exactly the same way.  It's rare for pkgs other
than MS's to attempt to use this mechanism, though (the reason is given
below).

> (Ignoring for now the fact that those reference counts don't
> always work!)

?  They work very well, in my experience.  Where they fail is when
installers & uninstallers break the rules.  MS publishes the list of MS DLLs
that are to be treated this way:  an installer "must" use refcounting on the
DLLs in the list.  Alas, some (especially older) installation pkgs don't.
Then the refcounts get screwed up.

That's what makes the mechanism brittle:  "the system" doesn't enforce it,
it relies on universal & intelligent cooperation.  It's very likely that
someone distributing a Python app will neglect (out of ignorance) to bump
the refcount on their Python components, so the refcount will be
artificially low, and a later uninstall of some unrelated pkg that *did*
follow the rules will merrily delete Python.

Gordon and I will repeat this until it sinks in <wink>:  almost everyone
with a successful Windows product ships the non-MS DLLs they rely on and
copies them into their own app directory.  It's simple and it works;
alternatives are complicated and don't work.  Many even ship & copy MS DLLs
(e.g., Scriptics copies its own msvcrt.dll (the MS C runtime) into Tcl's
directories).  Worrying about space consumed by redundant Python components
is a bad case of premature optimization <0.3 wink>.

> ...
> How can we do something similar for Python?

Seriously, short of getting MS to distribute Python and put the Python DLLs
on The List of refcounted resources, we should pursue this line reluctantly
if at all.  MS may have a better scheme in the future, but for now better
safe than sorry.


a-couple-mb-on-a-modern-pc-isn't-worth-the-time-it-took-
    to-read-this<wink>-ly y'rs  - tim