[Distutils] Disabling --single-version-externally-managed

[Toshio Kuratomi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Phillip J. Eby wrote:
> At 11:37 PM 9/1/2007 -0700, Toshio Kuratomi wrote:
>> I realize that taken in the vacuum of that single easy_install run,
>> require() works.  But the instructions are neglecting to tell the user
>> that things are more complex than that.  That depending on how the other
>> versions of the module are installed, pkg_resources may not work at all.
> 
> You're taking this out of context.  When you run "easy_install -m", the
> normal result is that *there is no default package*.  So require() is
> both necessary and sufficient in that case.
>
I'm saying that this fits into a larger context.  If we go forward with
using setuptools and eggs to manage multiple versions instead of coming
up with an ad hoc solution that changes the path ourselves, then
requires() is not sufficient.  So lacking documentation telling the end
user how to deal with this, we'll end up having to field more questions
about what to do in this situation.  This is not the situation we want
to be in.

> 
>>  Since __require__ works for this instance and for a mixture of -m and
>> - -s isn't it best to give users instructions that they can use
>> everywhere?
> 
> The problem is that you are adding a new situation to "everywhere", that
> previously didn't exist -- mixing single-version and multi-version at
> the system packaging level.
> 
Well, I'd argue that "everywhere" encompasses wierd situations that you
and I haven't even thought of yet that exist only on the planet Vogon
:-).  I'm just making use of your tool in a situation that seems logical
but hasn't been widespread until now.  So you can make a choice and say
"your situation is not the intended use of setuptools, please fork it or
come up with another way of achieving your ends" or figure out what, if
anything, is missing and help setuptools adapt to the situation.

> Either have a default version and deal with conflicts, or no default
> version and be explicit.

I'm trying to deal with conflicts.  And my understanding was that you
preferred people use __requires__ to do that.  In fact, you seemed to
say that __requires__ was the only way for people not using setuptools
to do that.

>  __requires__ is a workaround to avoid
> conflicts in specific cases.  It is intended only as a workaround, and
> really it's only for tools like easy_install and zc.buildout that
> generate scripts from a project description.
> 

1) Why a workaround?
2) What are the specific cases that it's limited to?

> I do not intend to support it for any other usage.  If you, as a
> packager, wish to package scripts that use __requires__, I don't see a
> problem with that.  It is emphatically *not* intended for general use.
> 
What is the real way to allow people to do quick and dirty scripting and
experimentation from the interpreter shell in this environment?

And once again, we are *not* creating and packaging scripts that use
__require__.  We are packaging multiple versions of modules with a
default.  We need to have instructions for end-users who want to do
quick and dirty scripting or experiment in the interpreter shell how to
select the version that they wish.

> There are already tools in place to do what you want; you're just trying
> to get away with not using them.
> 
No.  I'm trying to provide end users with an understanding of how to
work with the environment we are giving them.  We have no control over
what end users want to do so we have to give them enough information to
make an educated choice about how they can do the tasks they're used to.

> To put it another way, if you want to support multiple versions with a
> default, then you need to support the *end-user being able to choose*
> what version is the default.  

I'm trying to give end users a choice.  giving them a choice of defaults
is outside the scope of what I'm trying to accomplish but I'll be happy
if it works.

> If the user does an easy_install of some
> specific version, then *that* version needs to become the default.

This is fine with me but is really an extra level on top of what we're
trying to do.  We're working on system packaging.  The system packages
have to work together to give the user the ability to import MODULE and
the ability to get more specific than that.  If an end user
easy_installs something on top of the system packages it should "work"
with the packages installed on the system.  It doesn't matter to me, as
a system packager whether the end user decides to make it the default or
decides to make it explicitly requestable only as long as they are still
able to use their own version of the package if they so choose.

>  And
> the only way to support that, is for you to generate your scripts with
> easy_install or zc.buildout, or to make your own tool that generates
> scripts using __requires__.  If you don't, then your system-level Python
> scripts will be hosed if the user changes the default version of a
> package they use.
> 
Once again, *we are not creating any system level python scripts that
make use of a specific, non-default version*.  If we do, I'll be sure to
tell people to use easy_install to start their projects.

What we are doing is providing multiple module versions A) For end users
who need to use a different version in their scripts B) For projects
that have not been ported to a new version of the module.  We have no
control over how those upstream scripts are written -- whether they use
easy_install, hand code __requires__, or munge sys.path themselves.

> Now, it could be argued that you have packages that don't declare their
> dependencies currently (i.e., they don't use setuptools), and would also
> be hosed by a change in the default version.  But if that's true, then
> you're not fully supporting multiple versions.
> 
> So, my point here is that this issue is inherent to the nature of
> multiple versions.  Either you have a default, or you don't.  If you
> don't, you can't have conflicts but have to be explicit.  If you do have
> a default, then you need to *let the user change it*, and make
> everything you supply be explicit (so that it will automatically resolve
> conflicts).
> 
This is true.  But it's not going to change until all upstreams have
moved to using setuptools.  We provide a set of packages that work
together.  And we provide a means for the user to shoot themselves in
the foot.  But this is nothing new, they've had access to PYTHONPATH,
su, rm -rf, etc in the past.

What we can do is make as much of what they expect to work work as it
used to.  And where it differs, we offer a standard, logical, and simple
method to get the behaviour they want.

> I don't plan to support __requires__ for end-user scripts, but I'm happy
> to co-ordinate with system-level packagers and the authors of packaging
> tools to make sure you don't get messed up when it (eventually) goes
> away and is replaced by something better.
> 
I don't have too much of a problem with things changing.  But that is
also why I'm pushing so hard for this to be documented.  1) We'll want
to know when this changes and do the upgrade between releases of our
distribution with announcements of the change.  That way people will
know, in Fedora 80 we use __requires__ to request a specific version
from setuptools, in Fedora 81 we use the new and vastly superior
technique.  2) By being documented upstream, people won't accuse us of
relying on a private method that no one is supposed to use.  It'll
simply be one piece of API being replaced with another.

If you feel that __requires__ is a private method that shouldn't be used
then we'd respect that.  But between that and your teelling us that
pkg_resources.require() isn't meant to work like that, it means that
there's simply no method for making setuptools do what we need it to and
we'll have to find some other method to accomplish this.

- -Toshio
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFG2xlEX6yAic2E7kgRAvOSAKCqsrWaqyiZSqexLcLWZf6XN4ruwgCfbV5t
aan/Oqi39w8sT3yO1eI98FE=
=0Yv2
-----END PGP SIGNATURE-----