On 7/12/05, Phillip J. Eby wrote:

At 10:32 PM 7/11/2005 +0100, Paul Moore wrote:

...

2. It would be useful if the -f option worked with a file: URL (or better still, a simple directory name), to handle local repositories.

A good idea, which I've been thinking about. I'll add it in the next release, such that if an argument to -f is not a URL, it gets checked for being an existing directory. If it's a directory, I'll scan the contents as if they were links on a webpage. If it's not an existing directory, I'll print a warning (since errors should not pass silently) and proceed.

Agreed. Thanks.

...

3. I still dislike the fact that I can't get a listing of installed eggs.

I suggest writing a simple script using pkg_resources.find_distributions(), and perhaps adding some documentation and contributing it to setuptools. :)

...

And uninstalling by deletion isn't good, IMHO.

So write a pkg_delete script and contribute it, too.

See my earlier message. I will do so.

I've got no objection to these other tools, they just haven't been my top priority and aren't likely to be for a while. I'd like to focus on the core engine/API, in such a way that other people can easily write these sorts of management tools around it.

Great. I'll see what I can do to contribute in that area.

...

4. --dry-run doesn't work as expected: [...] Sadly, this is about what *I* expect, [...] I've been making the assumption that it's more useful to have a dry run that tells you most of what it does before dying, than to have no dry-run capability at all.

My reason for using --dry-run was because while I'm happy to do the build steps in a work area, I don't want to do the install until I see what will happen (just in case I've got something wrong!) So for me, dry-run capability is only really crucial for the install step. I'd actually done all of the previous steps (the build bits) for real in any case, so the dry run did nothing before failing... But I see what you mean, it's standard distutils machinery so you haven't got much freedom to change it, and ultimately when enough things aren't done, the remaining steps are meaningless anyway.

...

A nice to have feature would be the ability for easy_install to pick apart a bdist_wininst installer and build an egg from it.

Then I guess it's a good thing I implemented it already. :)

Cool! Now give back the time machine...

Try running EasyInstall on a win32.exe file, and you'll find that it actually does a pretty good job of turning it into an egg. If you were to run "easy_install pysqlite", for example, (assuming you could get past your firewall) you'd notice that it finds and downloads the bdist_wininst installer for PySQLite, converts it to an egg, and then installs it. This feature is only obscurely referenced in the docs and changelog, so I don't really blame you for missing it. :)

Quick question, is there a "convert but don't install yet" option? So I get an egg on my PC, but not in site-packages.

...

On a related note, had you thought of having EasyInstall put eggs in a specific subdirectory? Something like site-packages/eggs? As you're using .pth files, this wouldn't be any harder to manage, would it?

Yes, it would, actually, especially for non-system installs, as Michael Hoffman was asking about here recently. The main reason for putting eggs in sys.path directories, however, is to support automatic discovery and adding them to sys.path at *runtime*, dynamically. I also wanted to take advantage of certain properties of sys.path, like the fact that Python's start script has its directory added to sys.path. This then makes eggs in the script directory easily discoverable.

In fact, if you set an --install-dir with EasyInstall, and set the --always-copy flag, then your installation directory is an "application directory". Scripts and eggs are both copied there, including all eggs needed to run the application. Now you can just ship that directory and it can be used anywhere you have Python and setuptools installed.

Anyway, I think putting eggs in directories on sys.path is a natural extension of the sys.path concept, that seems to me to avoid the mental overhead of having a separate search path that works differently.

...

I can see a use case for a "combination" egg. One which packages together a group of packages. For example, if I want a standard set of packages installed on all machines, so that users can assume they are present when writing one-off scripts. If I could package *existing* eggs up into a combination site-std.egg

This is called a "basket" (as in putting all your eggs in one basket!), and it's actually already supported,

I told you - give back that time machine! :-) Seriously, I'm getting more and more impressed...

Anyway, if you end up with something nice and want to contribute it, that's great too. As with the other tools you've suggested, it's something I might get around to eventually, but there are some things I'm dying to get working first in the core engine, and I also have a lot of stuff I'd like to actually *use* the existing tools for right now. :)

Will do. I'm much closer to having a feel for where I'd be adding value and not duplicating effort, so I'll certainly see what I can do. Thanks for answering my dumb questions. Paul.

On 7/12/05, Phillip J. Eby wrote:

At 11:11 AM 7/12/2005 +0100, Paul Moore wrote:

...

Quick question, is there a "convert but don't install yet" option? So I get an egg on my PC, but not in site-packages.

Yes, it's called --install-dir, and you set it to the directory where you'd like it to dump the egg. :) If you don't want script wrappers in that directory too, use --exclude-scripts as well. So, "easy_install -xd somedir pysqlite" would dump the pysqlite egg in "somedir".

Neat, thanks. By the way, I don't know if the sqlite web page has changed, but your standard example of sqlite doesn't work for me:

\Apps\Python24\Scripts\easy_install.py -xd test sqlite Searching for sqlite Reading http://www.python.org/pypi/sqlite/ Reading http://pysqlite.org/ No local packages or download links found for sqlite error: Could not find distribution for Requirement('sqlite', [], ())

I tried a few others - cx_Oracle, ctypes, kinterbasdb, python-dateutil, and they didn't work either. PIL *nearly* did - it found the PIL 1.1.5 Windows installer for Python 2.4, but it then failed with error: c:\docume~1\gustav\locals~1\temp\easy_install-k1vu-u\PIL-1.1.5.win32-py2.4.exe is not a valid distutils Windows .exe which is probably wrong, as the URL was correct and the file is a Python installer, but I can't diagnose much further as the temp file was deleted - could it be binary/text file issues? Actually, if I download the file manually and do "easy_install.py -xd test PIL-1.1.5.win32-py2.4.exe" I still get the same error, so it's deeper than that... I can't think of much more I can do to diagnose this for you, but let me know if there's anything you'd like me to try. This is with setuptools-0.5a9, btw, I haven't had time to download the new version yet (there's no Windows installer, so I'll have to use the egg, which shouldn't be hard but I'd rather not compound the issues by using a mis-installed package :-)) Paul..

On 7/12/05, Kevin Dangoor wrote:

easy_install -U setuptools

is the easiest way to upgrade. And, once you have the new version, you will be able to do python -m easy_install -U setuptools to keep up to date.

That conflicts with a bdist_wininst installation. Shouldn't be too hard to sort out, but there's a little bit of a chicken and egg issue here :-) Actually, ez_setup.py works fine on Windows - so that may well be a better option for Windows users with network access. However, if setuptools is installed as an egg, python -m easy_install doesn't work (and putting the egg on PYTHONPATH, or even renaming it as .zip didn't help - this may be a bug in Python's -m support...) Paul.

On 7/12/05, Phillip J. Eby wrote:

...

But even then, things are going odd, as if I try to build an egg for ctypes (which isn't zip-safe) all of the files and subdirectories in the egg directory are missing the first two characters... Hmm, it happens with zip-safe eggs, as well, so I don't know - maybe it's related to the bdist_wininst unpacking code. Sorry, I can't debug this any further.

And I can't reproduce it. However, I have noticed that the format change for Python 2.4 includes a change to what data is in the 'cfgdata' string, and so I changed this line:

cfg.readfp(StringIO.StringIO(f.read(cfglen)))

to read:

cfg.readfp(StringIO.StringIO(f.read(cfglen).split(chr(0),1)[0]))

In order to ignore anything after the first NUL character in the cfgdata. I don't know if that was causing your two-character problem or not, as I can't reproduce it with Python 2.4 and the ctypes .exe for Python 2.4.

Looks like that fixed the problem. Thanks.

After the first thousand levels of recursion, the time machine throws a RuntimeError("Paradox stablization loop limit exceeded") and kicks me back to the present. :)

So *that's* why Guido resists implementing tail-call optimisation! :-) Paul

At 09:13 AM 7/13/2005 +0200, M.-A. Lemburg wrote:

...

It looks like the issue is in bdist_egg.py, write_safety_flag, where an ensure_directory() call is needed. There's a patch for this below, as well.

Talking about "safety": shouldn't this be addressed in a standard way, ie. signed packages ?

"Zip safety" refers to whether the package can be safely installed as a zip file; i.e., whether the package is likely to work once it has been installed that way. It's not about "safety" in some security sense.

At the very least, I'd expect the downloader to compare an MD5 checksum stored in PyPI with the one from the downloaded file. Of course, using GPG and checking the signature based on the public key of the author would be even better.

At the moment, PyPI only stores MD5's and signatures for packages uploaded to PyPI itself, which is an extremely small minority of packages, so I haven't implemented this yet. However, easy_install runs fine on local files, so you can download and verify files before running easy_install on them. If somebody wants to contribute patches for MD5 and signing, that would certainly be nice.