
[Fred says the need for application-specific command-line options is not clear] My impetus for this was the need for specific module distributions to be told where to go looking for external C libraries. For instance, the setup.py for zlib would need to know where to find zlib.h and libz.a -- that is, how to generate a -I option for compiling the extension, and a -L option for linking it (modulo the usual Unixisms). Likewise, tkinter needs to be told where to find the Tcl and Tk headers and libraries. Granted, there's already a standard way of doing this: the Setup file. Perhaps we should preserve the Setup file, or perhaps we should mutate it into a Distutils-specific 'options' file as per Martijn's suggestion for replacing my command-line options. However, let's not limit ourselves to "where's this header file, where's that library?" questions: lots of modules could have configuration options that the builder might want to supply at build time. And keep in mind that ultimately we want to be able to write setup.py for a whole Python *applications*, which almost certainly will have configuration options: where to put temporary files? permanent data files? installation directory? etc. [Martijn Faassen berates me for forgetting about abstract compiler classes]
Duh, you're right. I had *Programming Perl* sitting on my lap and open to the section on MakeMaker as I wrote that part of the design proposal. (Mainly I wanted to be sure I didn't forget anything obvious.) Small wonder a few Unixisms creeped in! Anyways, this raises the broader question: how should the class(es) that abstract compilation and linking behave? I'll try to come up with a few ideas and post them separately. (Arg, really must look at Interscript's compilation stuff... presumably John has already thought through this stuff!) Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 x287 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

Greg Ward writes:
My impetus for this was the need for specific module distributions to be told where to go looking for external C libraries. For instance, the
Why do these need to be application specific? Appropriate options can be defined (such as -I and -L;) to do this without making them application specific.
These should be easily handled using command-specific options, which may or may not need to be application-specific: A few basic options might be useful in the standard implementation simply so that the option names are consistent (where to put temp files, config files, etc.). I'm inclined to think we need to elaborate on the command dispatch mechanics a little to make sure we do the right thing to support both application-specific extensibililty and site-specific extensibility. I'll make a mini-proposal: Each command should be defined by a class. What class that is is determined using a registry initialized from a configuration file for the site (to allow site administration hooks) and the contents of a distutils sub-package. The application can update the registry as needed. This might look like this: ------------------------------------------------------------------------ import distutils.registry baseclass = distutils.registry.get_cmd_class("install") class DerivedInstallCmd(baseclass): LONG_OPTIONS = baseclass.LONG_OPTIONS + ["enable-foo", "disable-foo"] use_foo = 0 def __init__(self, options): for item in options[:]: opt, arg = item if opt == "--enable-foo": self.use_foo = 1 options.remove(item) elif opt == "--disable-foo": self.use_foo = 0 options.remove(item) baseclass.__init__(self, options) def run(self): # use self.use_foo somehow... distutils.registry.set_cmd_class("install", DerivedInstallCmd) ------------------------------------------------------------------------ The command dispatcher can use the same distutils.registry.get_cmd_class() function to get the class that should actually be used to implement the command, allowing all the "hooked-in" functionality to remain. Comments? -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191

Greg Ward wrote:
I think dependencies on external packages should be abstracted as much as possible. Otherwise extension/application writers still end up having to write specific configuration stuff for each platform Python runs on, which wasn't what we wanted. Perhaps we can distribute 'external package info files' for common external packages, describing how this package works on various platforms (where the files are, what they're called, how to link to them, etc). These external package info files should be independent of the Setup scripts of specific applications. This way someone else than the extension writer can maintain them, and you only need to do write an information file once for each external resource. In the end the writers of libraries may start making these info files by themselves. :)
And I think options file versus command line options warrants some more discussion, too. Not a whole flamewar, but I wouldn't like to settle on a vast slew of command line options if a nicely commented options file can be used instead. Obviously I'm biased more towards options files, see my earlier discussion in the thread. [snip]
Actually John Skaller and I and others had some dialogue on this topic on this list some time ago. John posted some relevant code, I posted some comments on this (if I didn't reply to sender instead of to list, as I'm about to do now, but I caught myself just in time :) If you want I can probably drag it up in my mailbox. I also posted an early design of what I thought it should look like, and what the requirements are. Not-berating-though-just-reminding-ly yours, Martijn

Martijn Faassen writes:
This is interesting. It sounds like we can go a long with something like this. It is interesting to note that gtk+ installs a script "gtk-config" that allows other makefiles to use something like: CFLAGS=`gtk-config --cflags` LDFLAGS=`gtk-config --libs` This is probably not the right approach for most packages, but it's worth being aware of. Perhaps we could supply (possibly as part of distutils) a script py-config that takes essentially the same arguments as gtk-config, but also a --package to specify what package we're actually interested in, or require a positional parameter that specifies the package: py-config --package zlib --version - or - py-config --version zlib Maintaining a sufficiently large collection of package info files may be difficult, but it's hard to tell. It may be feasible to use native package management databases on systems that use them; something like RPM may provide enough information, but I don't know enough about SysVR4 PKG system yet.
information file once for each external resource. In the end the writers of libraries may start making these info files by themselves. :)
Ideally, installed on package installation.
I don't see any reason both can't be used. Command line options could be used to override the options file if necessary. This may point to a stronger approach than getopt to the option parsing issue. -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191

Greg Ward writes:
My impetus for this was the need for specific module distributions to be told where to go looking for external C libraries. For instance, the
Why do these need to be application specific? Appropriate options can be defined (such as -I and -L;) to do this without making them application specific.
These should be easily handled using command-specific options, which may or may not need to be application-specific: A few basic options might be useful in the standard implementation simply so that the option names are consistent (where to put temp files, config files, etc.). I'm inclined to think we need to elaborate on the command dispatch mechanics a little to make sure we do the right thing to support both application-specific extensibililty and site-specific extensibility. I'll make a mini-proposal: Each command should be defined by a class. What class that is is determined using a registry initialized from a configuration file for the site (to allow site administration hooks) and the contents of a distutils sub-package. The application can update the registry as needed. This might look like this: ------------------------------------------------------------------------ import distutils.registry baseclass = distutils.registry.get_cmd_class("install") class DerivedInstallCmd(baseclass): LONG_OPTIONS = baseclass.LONG_OPTIONS + ["enable-foo", "disable-foo"] use_foo = 0 def __init__(self, options): for item in options[:]: opt, arg = item if opt == "--enable-foo": self.use_foo = 1 options.remove(item) elif opt == "--disable-foo": self.use_foo = 0 options.remove(item) baseclass.__init__(self, options) def run(self): # use self.use_foo somehow... distutils.registry.set_cmd_class("install", DerivedInstallCmd) ------------------------------------------------------------------------ The command dispatcher can use the same distutils.registry.get_cmd_class() function to get the class that should actually be used to implement the command, allowing all the "hooked-in" functionality to remain. Comments? -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191

Greg Ward wrote:
I think dependencies on external packages should be abstracted as much as possible. Otherwise extension/application writers still end up having to write specific configuration stuff for each platform Python runs on, which wasn't what we wanted. Perhaps we can distribute 'external package info files' for common external packages, describing how this package works on various platforms (where the files are, what they're called, how to link to them, etc). These external package info files should be independent of the Setup scripts of specific applications. This way someone else than the extension writer can maintain them, and you only need to do write an information file once for each external resource. In the end the writers of libraries may start making these info files by themselves. :)
And I think options file versus command line options warrants some more discussion, too. Not a whole flamewar, but I wouldn't like to settle on a vast slew of command line options if a nicely commented options file can be used instead. Obviously I'm biased more towards options files, see my earlier discussion in the thread. [snip]
Actually John Skaller and I and others had some dialogue on this topic on this list some time ago. John posted some relevant code, I posted some comments on this (if I didn't reply to sender instead of to list, as I'm about to do now, but I caught myself just in time :) If you want I can probably drag it up in my mailbox. I also posted an early design of what I thought it should look like, and what the requirements are. Not-berating-though-just-reminding-ly yours, Martijn

Martijn Faassen writes:
This is interesting. It sounds like we can go a long with something like this. It is interesting to note that gtk+ installs a script "gtk-config" that allows other makefiles to use something like: CFLAGS=`gtk-config --cflags` LDFLAGS=`gtk-config --libs` This is probably not the right approach for most packages, but it's worth being aware of. Perhaps we could supply (possibly as part of distutils) a script py-config that takes essentially the same arguments as gtk-config, but also a --package to specify what package we're actually interested in, or require a positional parameter that specifies the package: py-config --package zlib --version - or - py-config --version zlib Maintaining a sufficiently large collection of package info files may be difficult, but it's hard to tell. It may be feasible to use native package management databases on systems that use them; something like RPM may provide enough information, but I don't know enough about SysVR4 PKG system yet.
information file once for each external resource. In the end the writers of libraries may start making these info files by themselves. :)
Ideally, installed on package installation.
I don't see any reason both can't be used. Command line options could be used to override the options file if necessary. This may point to a stronger approach than getopt to the option parsing issue. -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191
participants (3)
-
Fred L. Drake
-
Greg Ward
-
Martijn Faassen