At 03:07 PM 8/28/2005 -0500, Ian Bicking wrote:

I don't know if setup.py is the right frontend for all of these. One specific issue is that one command creates the basic framework of a package, including a setup.py file. But maybe setuptools could be extended to support two kinds of entry points: one that is applied globally (what it currently has), and one that is applied only on demand (for framework-specific commands).

My thought is that at some point (0.7?) setuptools will have a 'nest' script that runs general-purpose commands, and it will have its own entry point group for commands to install, list, uninstall, etc.

At 06:32 PM 8/28/2005 -0500, Ian Bicking wrote:

Phillip J. Eby wrote:

...

At 03:07 PM 8/28/2005 -0500, Ian Bicking wrote:

...

I don't know if setup.py is the right frontend for all of these. One specific issue is that one command creates the basic framework of a package, including a setup.py file. But maybe setuptools could be extended to support two kinds of entry points: one that is applied globally (what it currently has), and one that is applied only on demand (for framework-specific commands).

My thought is that at some point (0.7?) setuptools will have a 'nest' script that runs general-purpose commands, and it will have its own entry point group for commands to install, list, uninstall, etc.

I suppose this would be a good place for a command to setup a new project (since that's kind of package management -- at least chronologically); would it be an appropriate entry point for other more project-specific commands? It doesn't seem like it to me; but then setup.py seems appropriate for those commands.

Yep.

I think for the command plugins, I'd imagine:

setup( .... setup_plugins=['spec1', 'spec2', ...])

Which writes to .egg-info/setup_plugins.txt (one line per requirement). Then it includes any 'distutils.extra_commands' entry points from those eggs, which act exactly like distutils.commands.

I'm confused. If you want to add custom commands to a particular project's setup.py, you just supply the standard distutils 'cmdclass' argument to setup, e.g.: setup( ... cmdclass = {'mycommand':mycommand} ) Conversely, if what you mean is that you'd like to ensure that certain distutils extensions are available when setup.py runs, then you should use the 'setup_requires' argument (see the setuptools docs).

Or maybe it could be something in setup.cfg? I'd actually find this sort of thing quite awkward in setup.py, since setup.py can't be (reasonably) written to.

I'm still not following you. Configuration for custom commands of course can always be put in setup.cfg.

Incidentally, is it possible to put setup.cfg in the .egg-info directory or somewhere?

No; that file belongs to the distutils, and the distutils decree that it's found in the setup.py directory.

At 06:58 PM 8/28/2005 -0500, Ian Bicking wrote:

Phillip J. Eby wrote:

...

Conversely, if what you mean is that you'd like to ensure that certain distutils extensions are available when setup.py runs, then you should use the 'setup_requires' argument (see the setuptools docs).

I'm thinking of framework-specific commands, not purely project-specific.

Then use setup_requires to list the eggs that offer those commands.

For instance, if I have a project that uses SQLObject, I would like to make the SQLObject admin commands available. This way I could do things like "python setup.py sqlcreate", "python setup.py sqlstatus", etc. I want a single setting in setup.py to add all the commands from SQLObject. But I also don't want every setup.py on the system to have these same commands, because they mean nothing for a project that doesn't use SQLObject.

Yep, this is exactly what the setup_requires keyword is for. If SQLObject is the egg with entry points for those commands, then setup_requires=['SQLObject>=0.7b1'] or whatever will do the trick, to the point of downloading SQLObject to the setup directory and installing it there if need be. (Of course, if SQLObject is already on sys.path due to the easy_install.pth file, then you will get those commands in every project anyway, but oh well.)

At 10:38 AM 8/29/2005 -0500, Ian Bicking wrote:

Phillip J. Eby wrote:

...

...

For instance, if I have a project that uses SQLObject, I would like to make the SQLObject admin commands available. This way I could do things like "python setup.py sqlcreate", "python setup.py sqlstatus", etc. I want a single setting in setup.py to add all the commands from SQLObject. But I also don't want every setup.py on the system to have these same commands, because they mean nothing for a project that doesn't use SQLObject.

Yep, this is exactly what the setup_requires keyword is for. If SQLObject is the egg with entry points for those commands, then setup_requires=['SQLObject>=0.7b1'] or whatever will do the trick, to the point of downloading SQLObject to the setup directory and installing it there if need be. (Of course, if SQLObject is already on sys.path due to the easy_install.pth file, then you will get those commands in every project anyway, but oh well.)

That last part is exactly what I'm trying to avoid. If every framework starts publishing 1-5 distutils commands -- and I think most frameworks have that potential -- it's going to be a bit overwhelming. And I don't like that installing SQLObject will needlessly change the entire system's setup.py commands.

If packages listed in setup_requires had extra commands loaded, that would work fine.

Personally, I think the simple way to deal with this is to package distutils extensions as separate packages; e.g. SQLObjectSetup or some such.

Ian Bicking wrote:

Phillip J. Eby wrote:

...

...

If packages listed in setup_requires had extra commands loaded, that would work fine.

Personally, I think the simple way to deal with this is to package distutils extensions as separate packages; e.g. SQLObjectSetup or some such.

Is there a way to force packages to be installed multiversion (from the package metadata itself)? I'm fine putting it in a seperate package, but I don't want to require people to install that package in a special way -- especially since it's kind of hard to do with easy_install anyway, when a requirement is installed implicitly. Maybe something in setup.cfg will do it?

Though I'll note this is also somewhat awkward -- for instance, looking at buildtools,I think some of those commands should be globally installed, and some shouldn't. But changing that means refactoring the commands into two distributions; inter-distribution refactoring feels very heavy to me, especially for something I'd consider a UI adjustment. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org

Kevin Dangoor wrote:

On 8/29/05, Phillip J. Eby wrote:

...

Personally, I think the simple way to deal with this is to package distutils extensions as separate packages; e.g. SQLObjectSetup or some such.

Could this type of thing be done with a "feature"? That way, it's still ultimately part of SQLObject, but people can choose to leave it out if they wish...

Entry points can require features, but AFAIK they can't be feature-specific. Unless if an entry point depended on a feature, and that feature wasn't loaded (which they aren't by default, even when the egg is installed for implicit use without require()), and the entry point wasn't loaded as a result. However, I think the entry point will be found either way, and then the feature required when the entry point is actually used. If there was a way to say that an entry point is only available when a feature is loaded (as opposed to saying that a feature must be loaded before using the entry point), then that would work. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org

On 8/29/05, Ian Bicking wrote:

I expect I'll still want a entry point for all these that acts like distutils.commands, but under a different name. Then a command-packages managing command (that's hard to parse ;) could use that to list available command packages. The command-packages managing command itself would be installed globally.

I have to wonder, though, how many commands is too many? And how many people will likely install enough packages to have that many commands? I, personally, was really happy with how easy and transparent the use of an EntryPoint was for allowing people to use Testido's distutils command. I find it hard to imagine that there would be a truly unmanageable set of commands in a typical person's installation. Kevin

At 10:04 PM 8/29/2005 -0500, Ian Bicking wrote:

My vision of how it would work with command-packages would be like:

python setup.py plugins --list # get a list of installed and uninstalled plugins, using entry points # to discover commands python setup.py plugins --remove=EggName python setup.py plugins --add=EggName # these change setup.cfg

I'm not sure whether you'd specify eggs, eggs+entry points, modules, or what. Or any of the above. I still have to remind myself how the underlying stuff works.

I'm not clear on what you're trying to do exactly. However, keep in mind that distutils commands running from setup.py aren't the only way to skin this cat. You can always define entry points in a different group than 'distutils.commands'. For example, when I create the 'nest' script, I'll probably have a 'nest.commands' entry point group that will be searched for commands. Such "non-setup" scripts can always use setuptools.sandbox to run a setup script and thus get access to setup metadata if they need to. So, if you need some kind of "dbadmin" tool for SQLObject, you can always have it use its own command entry points stuff, and then invoke setup.py with e.g. the --command-packages option. Conversely, if you want to enable certain setup.py commands based on the availability of specific setup() metadata, there's already a facility for that. Register an argument validator in 'distutils.setup_keywords', and if the metadata you want is present, it will be called. Your hook can then do: dist.cmdclass.setdefault('cmdname',command_class) to register the additional command(s). Argument hooks are called while the distribution object is being created, so this will be before the distutils has done anything with the commands. Note that you can't use this technique to *remove* commands that already exist in the distutils or setuptools, only to optionally-add them. Anyway, for commands that depend on non-standard setup metadata, this is an ideal way to hook them in, since you'll need a distutils.setup_keywords entry point to do argument validation anyway. So if the custom metadata passes validation, just register the additional command class(es). Thus, if you have setup() arguments for say, a DB schema, then you could automatically enable dbcreate/dbdrop commands for the setup script in that case. If the script doesn't supply the keywords for schema information, your keyword entry point won't get imported, and so the commands won't get added to the list for the setup script.

At 12:22 PM 8/30/2005 -0500, Ian Bicking wrote:

Interesting, I'll have to try that out. So I'm thinking I might be able to have someone add this to setup.cfg:

[sqlobject] dbmodule = mypackage.db

And use that to kind of implicitly enable the SQLObject commands. At least, assuming I can access arbitrary sections of the config from the setup (since sqlobject would imply several commands). Or alterately in setup.py. Hrm... maybe it has to be in setup.py, though, since setup.cfg is only used after the command is invoked, so it can't keep the command from being available? Well, I suppose experimentation is in order.

setup.cfg is loaded and parsed before the command line is, but *after* argument processing. So there isn't really a way to hook into any of that, short of loading the config file yourself from a keyword argument processor. It'd be better for you to define a setup() keyword like 'sqlobject_dbmodule'. See the CVS version of setuptools.txt under "Adding ``setup()`` Arguments" for detailed info. The setup() would also have to list 'setup_requires="SQLObject>=0.7b1"' or whatever in order to ensure that the keyword argument is recognized, though. So, the setup might look like: setup( ... setup_requires=["SQLObject>=0.7b1"], sqlobject_dbmodule = "mypackage.db", ) And there you go. Your "sqlobject_dbmodule" entry point in the "distutils.setup_keywords" group will then define a validation function to check that sqlobject_dbmodule has the right type/syntax, and then insert any useful command classes into the dist.cmdclass dictionary, making them available before command line parsing takes place.