[Distutils] The problem with Setuptools on Python 3.

[Lennart Regebro]

On Tue, Apr 21, 2009 at 14:15, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> I don't understand why not, doing it may be not entirely trivial but it
> should be possible with some trickery.  As PJE noted one way to do this is
> to explicitly convert setuptools to python 3.x syntax before actually
> running setup.py (e.g. his setup3.py file).  With some care this could even
> be done in setup.py itself.

I'm sorry, I don't think I'm able to explain it more in detail than
what I did in the first post in this discussion, unless you are more
explicit about what I need to explain.

Here it is again, for convenience:

"Currently for the Python 3 version, I've made a script that copies the
source tree and runs 2to3 on it. Then you can use that new tree to run
tests, install, make eggs and releases etc. The problems with this is:

a) The script is made for my computer only. It wouldn't work on
Windows, and it requires you to have lib2to3 checked out in a
particular place (although I think that's only if you use 3.0.0, and
not 3.0.1, but I haven't tested).

b) It's slow, as it copies all files, even those who hasn't changed
when you fixed a bug. Making a script that only copies the changed
files and runs 2to3 on them is possible, but I'd like not too, it
takes time and is likely to break. It seems like a waste.

Obviously, this is exactly the things distutils are meant to solve.
And indeed, distutils in Python 3 has a command called build_2to3
which solves exactly these problems. And indeed, this is the technique
I've used for the Python 3 branch of zope.interfaces. There I just run
setup.py install, and if I'm doing that under Python 3, it will first
run 2to3 on the code before installing. Same thing with running
setup.py test. It will sync changes to build/, run 2to3 on changed
files and then run the tests in build. It makes porting to Python 3
much easier, and it makes installing from source painless.

So why don't I use that for setuptools? Well, because:

c) The setup of setuptools requires setuptools. So to be able to do
the 2to3 conversion in the setup, I need to first convert the source
with 2to3. Yes, catch 22.

I've tried to get around this problem, but failed. Solutions I tried was:

I) Fall back to distutils if setuptools can't be imported. This means
that I can with some effort make a setup that will work under Python 3
and install setuptools under Python 3. Problem solved? No. Because
running setup.py again will just mean that it tries to import
setuptools from the local Python2 location, and it will fail. So in
this case I can't run tests of build eggs for setuptool.

II) Moving the source into a src directory. This means that when
setup.py runs, it will use the version of setuptools installed under
I), and building eggs etc is possible (I think, I haven't tested that
the eggs are correct). Problem solved? No, because you still can't
test it. Which leads us to problem d:

d) When running the tests, the setuptools module will already be
imported. But it will have imported the one in site-packages, *not*
the one in src, and it will therefore test the one in site-packages.
And that one doesn't have the api_tests.txt copied in, so that will
fail, and

e) even if api_tests.txt was copied, this would mean you had to
install setuptools before you test it"


-- 
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
+33 661 58 14 64