Re: [Distutils] The problem with Setuptools on Python 3.
At 09:11 PM 4/20/2009 +0200, Lennart Regebro wrote:
"Isolated"? What do you mean?
Making a separate setup script for Python 3, at least for setuptools itself, if not having a general convention for that, since other packages may want to ship 2+3 stuff in the same package. Or, in the alternative, using version testing in setup.py to run an alternate script for Python 3.
If you have suggestions on how to solve the problems I'm describing, can you please come forward with them, I've asked several times already. All you do is say "no, no, no, no, no", and "becasue". It isn't helpful.
I'm sorry you feel that way, as I've been *trying* to help. I just still don't get what the problem is. If I were porting setuptools to Python 3, I would *want* it to be circular, even if I had to hack on it a little at first. So I have a hard time understanding why you don't. But I'm not you -- I understand the code base and I'm not trying to port it. Maybe if I were trying to port it, I would get what problem you're having, or maybe I would just keep right on going and not notice. I don't know. I've been trying to find out what exactly is stopping you and just can't seem to wrap my brain around it, any more than you've been able to about the reverse.
For end-user convenience. A large number of people installing setuptools are not installing it because they are personally interested in it, but because something else they want uses it.
Yes? And again you don't explain how this leads to you conclusion.
Mostly because your questions aren't pinpointing what you want to know. You ask general questions, so I give general answers. We seem to both be suffering from a surplus of assumptions of what should be "obvious" to the other person. For example, this:
Eggs are not easier to install, on the contrary, I have tried and failed a couple of times, and ended up using the source install instead.
...seems to indicate that your question was actually about eggs, not other-than-source distributions in general. I was mostly talking about the wininst installers and source RPM. The eggs are there, on the other hand, for ez_setup.py to download. (Not to mention buildout's bootstrap script, and other tools that depend on setuptools and want to have an automated overall install process.)
On Mon, Apr 20, 2009 at 21:52, P.J. Eby <pje@telecommunity.com> wrote:
Making a separate setup script for Python 3, at least for setuptools itself, if not having a general convention for that, since other packages may want to ship 2+3 stuff in the same package.
The typical setup script will look exactly the same under python 2 and python 3. There is no need for separate scripts in the general usecase. If you want to run 2to3 automatically, all you need to do is set up build_py_2to3 instead of build_2to3, that's the only difference, and that's easily fixed with a importexception. This goes pretty much for setuptools also. The setup3.py script will more or less work under python2 as well.
Or, in the alternative, using version testing in setup.py to run an alternate script for Python 3.
You don't need alternative scripts. setuptools is an exception, because it depends on itself, providing a catch22 situation.
I'm sorry you feel that way, as I've been *trying* to help. I just still don't get what the problem is. If I were porting setuptools to Python 3, I would *want* it to be circular, even if I had to hack on it a little at first. So I have a hard time understanding why you don't.
But it CAN NOT be circular under Python 3.
Maybe if I were trying to port it, I would get what problem you're having, or maybe I would just keep right on going and not notice. I don't know.
You will have it and I explained in the mail I sent as a start of this discussion. If I was unclear, please tell me what you didn't understand.
I've been trying to find out what exactly is stopping you and just can't seem to wrap my brain around it, any more than you've been able to about the reverse.
Mostly because your questions aren't pinpointing what you want to know.
It is pinpointing them exactly. I want to know why setuptools need to depends on setuptools. Your answer, as I can understand it is for convenience, and so that it serves as a test and example of it's own features. The fact that is serves as a test of it's own features is another pain. That was a big reason for the difficulty of porting, as even when testcases all passed, not all features worked. So I have to say that although it sounds reasonable, I think it's misguided.
Eggs are not easier to install, on the contrary, I have tried and failed a couple of times, and ended up using the source install instead.
...seems to indicate that your question was actually about eggs, not other-than-source distributions in general.
Which probably is why I said eggs...
The eggs are there, on the other hand, for ez_setup.py to download. (Not to mention buildout's bootstrap script, and other tools that depend on setuptools and want to have an automated overall install process.)
OK, I wonder if there is a way around that. If not, then as far as I can see, there is no way to install or develop with setuptools smoothly in Python 3. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
On 20 Apr, 2009, at 22:12, Lennart Regebro wrote:
I'm sorry you feel that way, as I've been *trying* to help. I just still don't get what the problem is. If I were porting setuptools to Python 3, I would *want* it to be circular, even if I had to hack on it a little at first. So I have a hard time understanding why you don't.
But it CAN NOT be circular under Python 3.
I don't understand why not, doing it may be not entirely trivial but it should be possible with some trickery. As PJE noted one way to do this is to explicitly convert setuptools to python 3.x syntax before actually running setup.py (e.g. his setup3.py file). With some care this could even be done in setup.py itself. Ronald
On Tue, Apr 21, 2009 at 14:15, Ronald Oussoren <ronaldoussoren@mac.com> wrote:
I don't understand why not, doing it may be not entirely trivial but it should be possible with some trickery. As PJE noted one way to do this is to explicitly convert setuptools to python 3.x syntax before actually running setup.py (e.g. his setup3.py file). With some care this could even be done in setup.py itself.
I'm sorry, I don't think I'm able to explain it more in detail than what I did in the first post in this discussion, unless you are more explicit about what I need to explain. Here it is again, for convenience: "Currently for the Python 3 version, I've made a script that copies the source tree and runs 2to3 on it. Then you can use that new tree to run tests, install, make eggs and releases etc. The problems with this is: a) The script is made for my computer only. It wouldn't work on Windows, and it requires you to have lib2to3 checked out in a particular place (although I think that's only if you use 3.0.0, and not 3.0.1, but I haven't tested). b) It's slow, as it copies all files, even those who hasn't changed when you fixed a bug. Making a script that only copies the changed files and runs 2to3 on them is possible, but I'd like not too, it takes time and is likely to break. It seems like a waste. Obviously, this is exactly the things distutils are meant to solve. And indeed, distutils in Python 3 has a command called build_2to3 which solves exactly these problems. And indeed, this is the technique I've used for the Python 3 branch of zope.interfaces. There I just run setup.py install, and if I'm doing that under Python 3, it will first run 2to3 on the code before installing. Same thing with running setup.py test. It will sync changes to build/, run 2to3 on changed files and then run the tests in build. It makes porting to Python 3 much easier, and it makes installing from source painless. So why don't I use that for setuptools? Well, because: c) The setup of setuptools requires setuptools. So to be able to do the 2to3 conversion in the setup, I need to first convert the source with 2to3. Yes, catch 22. I've tried to get around this problem, but failed. Solutions I tried was: I) Fall back to distutils if setuptools can't be imported. This means that I can with some effort make a setup that will work under Python 3 and install setuptools under Python 3. Problem solved? No. Because running setup.py again will just mean that it tries to import setuptools from the local Python2 location, and it will fail. So in this case I can't run tests of build eggs for setuptool. II) Moving the source into a src directory. This means that when setup.py runs, it will use the version of setuptools installed under I), and building eggs etc is possible (I think, I haven't tested that the eggs are correct). Problem solved? No, because you still can't test it. Which leads us to problem d: d) When running the tests, the setuptools module will already be imported. But it will have imported the one in site-packages, *not* the one in src, and it will therefore test the one in site-packages. And that one doesn't have the api_tests.txt copied in, so that will fail, and e) even if api_tests.txt was copied, this would mean you had to install setuptools before you test it" -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64
Ronald Oussoren wrote:
On 20 Apr, 2009, at 22:12, Lennart Regebro wrote:
I'm sorry you feel that way, as I've been *trying* to help. I just still don't get what the problem is. If I were porting setuptools to Python 3, I would *want* it to be circular, even if I had to hack on it a little at first. So I have a hard time understanding why you don't.
But it CAN NOT be circular under Python 3.
I don't understand why not, doing it may be not entirely trivial but it should be possible with some trickery. As PJE noted one way to do this is to explicitly convert setuptools to python 3.x syntax before actually running setup.py (e.g. his setup3.py file). With some care this could even be done in setup.py itself.
But even if it's possible to make it circular, is that really a good design? I think not. Eric.
On 21 Apr, 2009, at 14:31, Eric Smith wrote:
Ronald Oussoren wrote:
On 20 Apr, 2009, at 22:12, Lennart Regebro wrote:
I'm sorry you feel that way, as I've been *trying* to help. I just still don't get what the problem is. If I were porting setuptools to Python 3, I would *want* it to be circular, even if I had to hack on it a little at first. So I have a hard time understanding why you don't.
But it CAN NOT be circular under Python 3.
I don't understand why not, doing it may be not entirely trivial but it should be possible with some trickery. As PJE noted one way to do this is to explicitly convert setuptools to python 3.x syntax before actually running setup.py (e.g. his setup3.py file). With some care this could even be done in setup.py itself.
But even if it's possible to make it circular, is that really a good design? I think not.
Given what setuptools tries to do I don't think the circular design is a problem. A standalone distutils would have had the same problem. In the long run the useful features of setuptools should incorperated into distutils, instead of monkeypatching distutils. That is, of course, an entirely different issue and one that's already being worked on. Ronald
Eric.
participants (4)
-
Eric Smith -
Lennart Regebro -
P.J. Eby -
Ronald Oussoren