[Distutils] easy_install: setup.py-less packages

Phillip J. Eby pje at telecommunity.com
Mon Jul 4 18:23:55 CEST 2005

At 03:51 AM 7/4/2005 -0400, Ryan Tomayko wrote:
>>>Phillip J. Eby wrote:
>>>>You then publish your wrapping package as a subversion URL.  This
>>>>work for auto-discovery from PyPI unless your URL ends with
>>>>Flup.egg or
>>>>something like that, but you can always give it to EasyInstall as
>>>>a URL,
>>>>or build eggs and publish those.
>>Ian Bicking wrote:
>>>Yes.  Hmm... I suppose I could create something that redirected
>>>from a
>>>detectable URL to the actual URL.  That's kind of hacky though.  Only
>>>being able to list actual eggs decreases the utility of a custom
>>>   Could it also look at title attributes or something?
>>>It would be neat if you could download a working development copy
>>>easy_install too.  I'd like to encourage, or at least assist,
>>>people who
>>>want to interact with the code itself; and if you use a checkout
>>>you can
>>>at least use things like svn diff, even if you can't necessarily
>Ryan Tomayko wrote:
>>That's interesting. If you could point to subversion repositories
>>from an index page, you could throw a page somewhere with a big
>>list of project subversion URLs that use some kind of convention on
>>   <a href="http://example.com/svn/foo/trunk/">foo-devel</a>
>>   <a href="http://example2.com/svn/bar/trunk/">bar-devel</a>
>That's going to be ambiguous though, right. How do you know those
>URLs are subversion repositories without having to go out and HEAD
>each resource or something.
>We can use the rel attribute:
>      <a href="http://example.com/svn/foo/trunk/" rel="subversion">
>      foo-devel
>      </a>

Um, yeah.  You guys probably haven't noticed this, but EasyInstall detects 
links using a regular expression that looks for "href="; it doesn't even 
try to parse HTML, and I don't really want it to start trying, 
either.  It'd be better if we could put the information in the URL itself, 
e.g. "http://example.com/svn/foo/trunk#egg=foo_devel".  IOW, I could have 
EasyInstall extract information from a specially-formatted "fragment" 
portion of a URL, which should be safe because it doesn't get transmitted 
to the server.  EasyInstall already knows when it fetches a subversion 
http: URL, and automatically does a checkout in that case.

Of course, the only current use case for such fragment identifiers is to 
mark a non-egg URL as a discoverable subversion checkout.  For this 
purpose, EasyInstall really needs to know the effective version number, 
unless it just treats subversion checkouts as being the highest possible 
version number for purposes of selecting a download target.  Either that, 
or the development checkout of a project needs to be treated as a project 
with a different name.  For example, the development project for "foo" 
could be "foo_devel", and therefore installing "foo_devel" gives you 
something different.  Or, there could be a command-line option to 
EasyInstall (--develop, perhaps) that requests subversion checkouts take 
precedence over other kinds of links (so that they don't have to have a 
version number in the link).  In this case, we could make the fragment 
syntax be something like "#svn=foo" to indicate that the link is a 
subversion checkout for the "foo" project.

Okay, so here's the idea.  I add #svn=projectname detection support, but 
treat such links as the lowest possible version number (an empty version), 
unless the --develop flag is supplied or the fragment includes a version 
number.  In this way, subversion links are always discoverable but not used 
unless they are either the only link and you have no version installed, or 
the person creating the link has identified the version.

But with the --develop flag, what happens is that instead of installing the 
package and its dependencies, you just check out the named package(s) to a 
directory (or directories) named for the project under(s) your designated 
--build directory.  It then runs a "setup.py develop" command on each of 
the checked-out projects to create links and script wrappers in your 
--install directory, so that you can use them.

The "develop" command (which doesn't exist yet, but which I'd originally 
planned to write this weekend) installs an .egg-link file in the target 
directory that points to the development source root, and runs 'build_ext 
-i' to build any C extensions in-place, so they'll run from the source 
directory.  Oh, and it also updates the project .egg-info so that it's 
usable as an egg, and puts stub scripts in the --script-dir that run the 
original source scripts after the appropriate 'require()' 
operation(s).  This then allows you to develop in one directory, while 
having that code "deployed" to another.  (In fact, you can have your 
development checkout deployed to multiple target locations at the same 
time, if desired.)  There would also be an --uninstall flag for the develop 
command to let you remove the wrappers and .egg-link, so that you could 
then install a non-development version to the target.

Anyway, a --develop option for EasyInstall could be added in order to run 
"setup.py develop" on checked-out development versions of packages you're 
working on, using EasyInstall's --install-dir and --script-dir as the base 
for the "develop" commands.

I'm also planning, by the way, to update setuptools' "test" command to 
support running from a built egg; right now it still does a traditional 
distutils "install" instead.

More information about the Distutils-SIG mailing list