At 07:57 PM 1/11/2008 -0800, Philip Jenvey wrote:

Jython trunk has made a lot of progress on setuptools support. The gory details are here:

http://wiki.python.org/jython/SetuptoolsOnJython

By the way, the reason setuptools uses -E for the subprocess is so that Python will ignore any environment variables such as PYTHONPATH et al. It's not clear to me whether making -E a no-op for Jython will actually make this work correctly. I also have some concerns about how the $py.class stuff will work with distutils in general as well as setuptools in particular, but I've tried as much as possible to make setuptools follow the distutils rather than do its own thing. So hopefully most of the relevant things will be taken care of by Jython-level distutils patches.

The final few issues require some changes to setuptools itself:

o usage of os.chmod. most uses are actually wrapped in an os.name == 'posix', or except AttributeError checks, but one was missed (Jython lacks os.chmod)

o a unit test assumed CPython dict ordering

I've attached a patch against the 0.6 branch for both of these issues.

Thanks!

Though there's one more issue that'll be more difficult to deal with: setuptools uses marshal.load to read variables (via co_names and co_consts) from a module's bytecode (pyc/o) without importing it. Jython and IronPython don't compile to .pyc, nor do their code objects support co_names and co_consts, so this isn't portable.

marshal.load is used in a couple places:

o setuptools.depends.extract_constant, which is only used in setuptools by Require.get_version(). I don't know when this is used or what it's used for

setuptools.depends is essentially an experimental pre-egg feature that shouldn't be used by anything that's supported or documented at the moment.

o to find un-zip_safe variables/calls when a distribution doesn't specify zip_safetyness. Probably the easiest way to deal with this is to just fallback to zip_safe=False on these platforms

That's probably reasonable. An alternative would be to use the tokenize module to read the source code, since all it really does is look at name tokens and string constants, which can both be identified via tokenization without any higher-level parsing.

At 05:57 PM 1/12/2008 -0800, Philip Jenvey wrote:

Attached is a simple patch to disable the zip_safe scan on Jython/ IronPython. Maybe in the future we can use tokenize when all of the source code is present in an archive.

I've now incorporated comparable changes for bytecode scanning and chmod in the trunk and 0.6 branches. For various reasons (including consistency of logging) I made somewhat more extensive revisions to chmod usage than was in your patch. I also implemented platform detection a bit differently. But the results should be the same. Please let me know if you have any further issues with setuptools on Jython. (I'm rather excited that it works there at all...)

If setuptools works on jython, what does it use for the "pyX.Y" part of the egg names? And in the future hopefully IronPython, PyPy, and ShedSkin will join the setuptools party. Does this revive our interest in supporting "100% Pure Python" eggs that do not have an X.Y Python version number in their names? (Since Python code without native extension modules is a lot more likely to work on various versions of various implementations of Python than Python code with native extension modules.) Or will jython, IronPython, PyPy, and ShedSkin specify some X.Y version of CPython that they support? Regards, Zooko

On Jan 22, 2008, at 10:13 AM, Tres Seaver wrote:

What is the advantage of such a 'noarch' egg over just distributing 'sdist' tarballs (AKA "source eggs")? Setuptools / easy_install is happy to consume them, and I would much rather work with them.

That's an excellent point. Honestly, the only reason I ever notice this arch question is because the setuptools egg itself (for bootstrapping with ez_setup.py) has a pyX.Y in it, and this makes it cost an extra 600 KB if I want to bundle setuptools with my app. I can solve this easily enough by hacking my copy of ez_setup.py to load a setuptools egg without an X.Y in its name, though... ;-) So I withdraw my interest in no-arch eggs, because you've reminded me that I strongly prefer source eggs. Regards, Zooko

On Jan 22, 2008, at 8:35 AM, zooko wrote:

If setuptools works on jython, what does it use for the "pyX.Y" part of the egg names?

And in the future hopefully IronPython, PyPy, and ShedSkin will join the setuptools party.

Does this revive our interest in supporting "100% Pure Python" eggs that do not have an X.Y Python version number in their names? (Since Python code without native extension modules is a lot more likely to work on various versions of various implementations of Python than Python code with native extension modules.)

Or will jython, IronPython, PyPy, and ShedSkin specify some X.Y version of CPython that they support?

You bring up a good point actually; setuptools bdist_egg hardcodes pyX.Y for egg filenames, so Jython currently creates eggs tagged as - py2.3. This is going to be odd because eggs created from Jython don't contain .pyc files, instead they contain Jython $py.class files. In the case of a zip_safe=False egg, easy_install actually re- compiles the eggs' .py files to byte code during their installation. So a zip_safe=False egg created from Jython would end up with .py, .pyc and $py.class when installed on CPython. Though it looks like no byte compiling is done for zip_safe=True eggs. These eggs aren't extracted and recompiled, they're just installed as is. In that case an egg created from Jython would end up installed on CPython with no .pyc files. CPython would fallback to using the .py file in that case, but that's not ideal. Is it worth it to use a different filename pattern than pyX.Y on Jython? -- Philip Jenvey