I'd like to get rid of the "paster" script, moving some of what it does into setup.py or elsewhere. To a degree that is possible, but I think requires some additions to setuptools: * An entry point group that is not globally looked up, like distutils.commands. This would be a set of commands the package provides only for itself. I expect it to look like: [distutils.commands.local] sql_record = sqlobject.manage.distextension:sql_record Exactly like what we have currently, just not looked up globally. * Additionally, or probably even better, something that enumerates locations for commands. E.g.: extra_commands=['SQLObject'] And then it would look in the SQLObject egg for distutils.commands.local, and apply everything found there to this package. Right now, for instance, buildutils adds a "pytest" command to every project, even though it only applies to some projects (for some commands this is ok, like "info", so two different entry points makes sense). A project could list itself to provide its own custom commands; I think that won't be too circular. Typically commands you provide for yourself or someone else are different -- e.g., the SQLObject commands don't apply to SQLObject itself. * Everything that can be done on a deployed egg will probably go in app/egg-specific command-line scripts, and maybe I'll make a little framework to make these easier to write, but that's entirely an implementation detail. But I'm also thinking that extra_commands could be used as a hint to that framework, and would kind of facilitate coordination of in-development commands with after-deployment commands. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
Phillip -- any thoughts on this? If this went in, this would allow me to start splitting up and simplifying some commands, as outlined in: http://blog.ianbicking.org/paster-and-scripts.html Ian Bicking wrote:
I'd like to get rid of the "paster" script, moving some of what it does into setup.py or elsewhere.
To a degree that is possible, but I think requires some additions to setuptools:
* An entry point group that is not globally looked up, like distutils.commands. This would be a set of commands the package provides only for itself. I expect it to look like:
[distutils.commands.local] sql_record = sqlobject.manage.distextension:sql_record
Exactly like what we have currently, just not looked up globally.
* Additionally, or probably even better, something that enumerates locations for commands. E.g.:
extra_commands=['SQLObject']
And then it would look in the SQLObject egg for distutils.commands.local, and apply everything found there to this package. Right now, for instance, buildutils adds a "pytest" command to every project, even though it only applies to some projects (for some commands this is ok, like "info", so two different entry points makes sense). A project could list itself to provide its own custom commands; I think that won't be too circular. Typically commands you provide for yourself or someone else are different -- e.g., the SQLObject commands don't apply to SQLObject itself.
* Everything that can be done on a deployed egg will probably go in app/egg-specific command-line scripts, and maybe I'll make a little framework to make these easier to write, but that's entirely an implementation detail. But I'm also thinking that extra_commands could be used as a hint to that framework, and would kind of facilitate coordination of in-development commands with after-deployment commands.
At 12:47 PM 4/19/2006 -0500, Ian Bicking wrote:
Phillip -- any thoughts on this? If this went in, this would allow me to start splitting up and simplifying some commands, as outlined in: http://blog.ianbicking.org/paster-and-scripts.html
Ian Bicking wrote:
I'd like to get rid of the "paster" script, moving some of what it does into setup.py or elsewhere.
To a degree that is possible, but I think requires some additions to setuptools:
* An entry point group that is not globally looked up, like distutils.commands. This would be a set of commands the package provides only for itself. I expect it to look like:
[distutils.commands.local] sql_record = sqlobject.manage.distextension:sql_record
Exactly like what we have currently, just not looked up globally.
* Additionally, or probably even better, something that enumerates locations for commands. E.g.:
extra_commands=['SQLObject']
You can pass a 'cmdclass' argument to setup() to accomplish these things. Normally, it's expected to be a dictionary, but as long as you give it an object that implements 'in', 'get()', 'keys()' and setitem/getitem operations, you should be good to go. (The distutils and setuptools both cache their own lookups by assigning items back into the cmdclass object.) If you come up with a nice "command map" object that lets you do something like: setup( cmdclass = CommandMap( # configuration stuff here ), ... ) then it would make a nice addition to setuptools for people that need it.
A project could list itself to provide its own custom commands; I think that won't be too circular.
That's only 'cause you haven't tried it yet. Setuptools has to do this, sort of, and trust me, it's a bitch. :)
Typically commands you provide for yourself or someone else are different -- e.g., the SQLObject commands don't apply to SQLObject itself.
That doesn't matter; the problem is that until your egg_info is generated, you don't have a pkg_resources.Distribution in existence to provide the entry points.
* Everything that can be done on a deployed egg will probably go in app/egg-specific command-line scripts, and maybe I'll make a little framework to make these easier to write, but that's entirely an implementation detail.
I actually do want "nest" to provide a nice mini-framework for writing command-line apps as functions -- sort of a stripped-down cross between peak.running.commands and peak.running.options. One key feature will be that the commands have to accept an argument that's a logger for them to use, and the verbosity of that logger will be managed by the caller/framework, not the function. (Meaning that verbosity options will be handled by the framework when running via the command line.) Although perhaps I should just write this hypothetical framework as a separate package... but I want nest to be a part of the core setuptools distribution. I guess if you're going to depend on the command line framework, you might as well depend on setuptools, since you'll need to have setuptools in order to be able to depend on the hypothetical command line package. :) (Confusing, isn't it?)
Phillip J. Eby wrote:
At 12:47 PM 4/19/2006 -0500, Ian Bicking wrote:
Phillip -- any thoughts on this? If this went in, this would allow me to start splitting up and simplifying some commands, as outlined in: http://blog.ianbicking.org/paster-and-scripts.html
Ian Bicking wrote:
I'd like to get rid of the "paster" script, moving some of what it does into setup.py or elsewhere.
To a degree that is possible, but I think requires some additions to setuptools:
* An entry point group that is not globally looked up, like distutils.commands. This would be a set of commands the package provides only for itself. I expect it to look like:
[distutils.commands.local] sql_record = sqlobject.manage.distextension:sql_record
Exactly like what we have currently, just not looked up globally.
* Additionally, or probably even better, something that enumerates locations for commands. E.g.:
extra_commands=['SQLObject']
You can pass a 'cmdclass' argument to setup() to accomplish these things. Normally, it's expected to be a dictionary, but as long as you give it an object that implements 'in', 'get()', 'keys()' and setitem/getitem operations, you should be good to go. (The distutils and setuptools both cache their own lookups by assigning items back into the cmdclass object.)
It looks like, if I do this, it will also have to implement the current default behavior of finding distutils.commands providers. Which isn't a big deal as long as that doesn't change, I guess. Though cmdclass won't get a reference to the setuptools.dist.Distribution object, so some things might not be possible.
If you come up with a nice "command map" object that lets you do something like:
setup( cmdclass = CommandMap( # configuration stuff here ), ... )
then it would make a nice addition to setuptools for people that need it.
OK, I'll look into that. I'm guessing it'll be something like CommandMap([eggs]), and maybe some other options, though I can't think of any at the moment. Maybe also reading .egg-info/command_map.txt if not given any list of eggs -- I'd like to keep options for putting metadata into .egg-info/*.txt files instead of setup.py.
A project could list itself to provide its own custom commands; I think that won't be too circular.
That's only 'cause you haven't tried it yet. Setuptools has to do this, sort of, and trust me, it's a bitch. :)
Yeah... it probably isn't important anyway, I can't think of any cases where I'd really want to use this.
* Everything that can be done on a deployed egg will probably go in app/egg-specific command-line scripts, and maybe I'll make a little framework to make these easier to write, but that's entirely an implementation detail.
I actually do want "nest" to provide a nice mini-framework for writing command-line apps as functions -- sort of a stripped-down cross between peak.running.commands and peak.running.options. One key feature will be that the commands have to accept an argument that's a logger for them to use, and the verbosity of that logger will be managed by the caller/framework, not the function. (Meaning that verbosity options will be handled by the framework when running via the command line.)
In my code I do a lot of specific testing of verbosity, to try to avoid redudancy. For instance: def ensure_dir(dir): if verbosity >= 2: print 'Trying to make directory %s' % dir if os.path.exists(dir): if verbosity >= 2: print ' (directory already exists)' elif verbosity >= 1: print 'Directory %s already exists' % dir It's a little annoying to write this, but log-style output is fairly crappy; a better compromise might be: def ensure_dir(dir): log.debug('Trying to make directory %s' % dir) if os.path.exists(dir): log.debug(' (directory already exists)') log.message((log.INFO, log.DEBUG), 'Directory %s already exists' % dir) I think the features for a simple two-level command framework would be fairly simple: * Register commands somehow (entry points, I suppose). Commands have a name and aliases. * Commands can be called to run them, with all arguments after the command name. * Commands have a help() method or something like that, that returns a string help message. Probably two forms -- a long-form help, and short-form. Maybe only the short form? * The framework provides "cmd -h", "cmd help", "cmd -h subcommand", and "cmd help subcommand". The last two might just call "cmd subcommand -h" internally. * I suppose the framework needs to be given some global help message. * It should be possible to hide commands or aliases from help. E.g., commands that represent an internal interface, or aliases that are only provided for backward compatibility. * Maybe a standard way to activate specific eggs, e.g., "cmd --require='Foo==1.0' ...". Maybe also a way of controlling the PYTHONPATH, since setting environmental variables is not equally easy in all environments. Everything else should be possible to build from there with other sub-frameworks. Self-referencing entry point loading would be nice. Then, for instance, you could do: entry_points=""" [whatever.this.group.is] sql-setup = sqlobject.commands:setup_command """ If setup_command is called with the package's Distribution object, then it can read config files or whatever from the egg, and you don't have to create little bits of glue all over the place. Hmm... putting information into the entry point name is problematic here, because SQLObject would give a set of commands which would be a bit tedious to enumerate, and it would be better if you could introduce the whole set of commands with one entry point. Also, it doesn't resolve the problem of aliases. Though it does give you an opportunity to resolve naming conflicts by renaming a command.
Although perhaps I should just write this hypothetical framework as a separate package... but I want nest to be a part of the core setuptools distribution. I guess if you're going to depend on the command line framework, you might as well depend on setuptools, since you'll need to have setuptools in order to be able to depend on the hypothetical command line package. :) (Confusing, isn't it?)
Would setuptools stop using the distutils command framework? -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
At 03:14 PM 4/19/2006 -0500, Ian Bicking wrote:
It looks like, if I do this, it will also have to implement the current default behavior of finding distutils.commands providers.
No it won't. You should just look up things that you want to provide; setuptools and distutils will do their own thing if your __contains__ says False or your get() returns the default when they ask for a command class. They will then call your __setitem__ to cache whatever they find.
If you come up with a nice "command map" object that lets you do something like: setup( cmdclass = CommandMap( # configuration stuff here ), ... ) then it would make a nice addition to setuptools for people that need it.
OK, I'll look into that. I'm guessing it'll be something like CommandMap([eggs]), and maybe some other options, though I can't think of any at the moment. Maybe also reading .egg-info/command_map.txt if not given any list of eggs -- I'd like to keep options for putting metadata into .egg-info/*.txt files instead of setup.py.
Yecch. When I first started doing egg stuff for setuptools, I thought it'd be nice to just edit files in .egg-info, and then I quickly got sick of it. IMO, setup.py should be a one-stop-shop. It's easier to explain to setup authors, too. I can understand not wanting to go to the trouble of implementing an egg_info "writer" plugin, but trying to show people how to actually use these things, I think your users are a lot better off if all they have to do is pass in a setup keyword or two.
I think the features for a simple two-level command framework would be fairly simple:
* Register commands somehow (entry points, I suppose). Commands have a name and aliases. * Commands can be called to run them, with all arguments after the command name. * Commands have a help() method or something like that, that returns a string help message. Probably two forms -- a long-form help, and short-form. Maybe only the short form? * The framework provides "cmd -h", "cmd help", "cmd -h subcommand", and "cmd help subcommand". The last two might just call "cmd subcommand -h" internally. * I suppose the framework needs to be given some global help message. * It should be possible to hide commands or aliases from help. E.g., commands that represent an internal interface, or aliases that are only provided for backward compatibility. * Maybe a standard way to activate specific eggs, e.g., "cmd --require='Foo==1.0' ...". Maybe also a way of controlling the PYTHONPATH, since setting environmental variables is not equally easy in all environments.
Everything else should be possible to build from there with other sub-frameworks.
You left out configuration.
Self-referencing entry point loading would be nice. Then, for instance, you could do:
entry_points=""" [whatever.this.group.is] sql-setup = sqlobject.commands:setup_command """
If setup_command is called with the package's Distribution object, then it can read config files or whatever from the egg, and you don't have to create little bits of glue all over the place.
Hmm... putting information into the entry point name is problematic here, because SQLObject would give a set of commands which would be a bit tedious to enumerate, and it would be better if you could introduce the whole set of commands with one entry point. Also, it doesn't resolve the problem of aliases. Though it does give you an opportunity to resolve naming conflicts by renaming a command.
Sorry, I don't really understand what you're trying to do here. Are you talking about adding something to "nest", or...?
Would setuptools stop using the distutils command framework?
No - it's necessary for compatibility, and to handle the intricate interdependencies where some commands depend on values computed by other commands. Nest has much simpler needs that would be better served by less implicit coupling.
Phillip J. Eby wrote:
At 03:14 PM 4/19/2006 -0500, Ian Bicking wrote:
It looks like, if I do this, it will also have to implement the current default behavior of finding distutils.commands providers.
No it won't. You should just look up things that you want to provide; setuptools and distutils will do their own thing if your __contains__ says False or your get() returns the default when they ask for a command class. They will then call your __setitem__ to cache whatever they find.
OK.
If you come up with a nice "command map" object that lets you do something like: setup( cmdclass = CommandMap( # configuration stuff here ), ... ) then it would make a nice addition to setuptools for people that need it.
OK, I'll look into that. I'm guessing it'll be something like CommandMap([eggs]), and maybe some other options, though I can't think of any at the moment. Maybe also reading .egg-info/command_map.txt if not given any list of eggs -- I'd like to keep options for putting metadata into .egg-info/*.txt files instead of setup.py.
Yecch. When I first started doing egg stuff for setuptools, I thought it'd be nice to just edit files in .egg-info, and then I quickly got sick of it. IMO, setup.py should be a one-stop-shop. It's easier to explain to setup authors, too. I can understand not wanting to go to the trouble of implementing an egg_info "writer" plugin, but trying to show people how to actually use these things, I think your users are a lot better off if all they have to do is pass in a setup keyword or two.
Putting things in setup.py means that those things are pretty opaque for tools. For instance, when generating packages setup.py is a point of contention when you are creating a package that uses more than one framework. If the generator was able to handle the specific files like entry_points.txt with knowledge of what those files meant, most contention could be avoided. requires.txt is another thing that could be usefully manipulated with tools; for instance, if you had a tool that did commits to two projects at once, and updated the second project to require on the after-commit version of the first project. So it's not the writer plugin that bothers me at all, though I haven't looked at the details of that. It's that Python source is opaque. Anyway, another thing occurred to me. In addition to providing plain egg files, I'd like to allow egg+entry_point name to be a possible specifier. It would be nice if we had a convention for how to specify both at once. I'm using spec#entry_point_name in Paste, which works fine for me.
I think the features for a simple two-level command framework would be fairly simple:
* Register commands somehow (entry points, I suppose). Commands have a name and aliases. * Commands can be called to run them, with all arguments after the command name. * Commands have a help() method or something like that, that returns a string help message. Probably two forms -- a long-form help, and short-form. Maybe only the short form? * The framework provides "cmd -h", "cmd help", "cmd -h subcommand", and "cmd help subcommand". The last two might just call "cmd subcommand -h" internally. * I suppose the framework needs to be given some global help message. * It should be possible to hide commands or aliases from help. E.g., commands that represent an internal interface, or aliases that are only provided for backward compatibility. * Maybe a standard way to activate specific eggs, e.g., "cmd --require='Foo==1.0' ...". Maybe also a way of controlling the PYTHONPATH, since setting environmental variables is not equally easy in all environments.
Everything else should be possible to build from there with other sub-frameworks.
You left out configuration.
Configuration...? I don't see where that fits in? Or, at least, I figured that configuration would be figured out by each subcommand, and they'd share command-line options for that by convention. I did forget --version, though, which should be implemented in the parent command.
Self-referencing entry point loading would be nice. Then, for instance, you could do:
entry_points=""" [whatever.this.group.is] sql-setup = sqlobject.commands:setup_command """
If setup_command is called with the package's Distribution object, then it can read config files or whatever from the egg, and you don't have to create little bits of glue all over the place.
Hmm... putting information into the entry point name is problematic here, because SQLObject would give a set of commands which would be a bit tedious to enumerate, and it would be better if you could introduce the whole set of commands with one entry point. Also, it doesn't resolve the problem of aliases. Though it does give you an opportunity to resolve naming conflicts by renaming a command.
Sorry, I don't really understand what you're trying to do here. Are you talking about adding something to "nest", or...?
Well, to be more specific, here's what I'd like for SQLObject: * A setup.py command, like "setup.py sql-record". This would take the current database schema as defined by the source, and write that out somewhere into the project's source (kept in the source as a historical record of all possible schemas the app has had.) * The setup.py command has to figure out where to find SQLObject classes in the project, by some configuration. I'd rather that configuration not go in setup.cfg, because it is used later... * The SQLObject-using-application would get several subcommands: - create - drop - show-sql - status - upgrade * These commands would read some configuration from the original egg, to find the history of schemas, and to find the module that holds the current SQLObject classes. So, particularly in the second case, adding SQLObject to your application introduces a set of commands, and those commands can all run off the same configuration, which they also share with the setup.py command. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
participants (2)
-
Ian Bicking
-
Phillip J. Eby