Here's the short form of my question:
How do people generally keep their Python modules and any configuration data for those Python modules separate?
I'm looking for both conventions (e.g., config goes in /etc/foo) and mechanics (e.g., use distutils with options x, y, and z).
The longer form of the question is a bit more convoluted, so for those with patience, read on. ;-)
I'm transitioning from Windows to Linux. So part of my question is about the conventions for separating software from data on Linux. I assume it's self-explanatory why you'd want to separate software from data. In case it's not, here's why I think it's important to separate: You don't really have to backup software (although you can if you want), but you do have to backup data (it's not like you can re-download it). So, if push comes to shove, and you have limited resources/time, it's nice to be able to focus on the important stuff and only back that up.
There's always a gray line between software and data that I label configuration data. In the Linux world, I guess that'd constitute anything you do to install any piece of software beyond the equivalent of configure; make; make install. If you specify any configuration options, create daemon accounts, etc.--ideally you'd do that with a script and you'd store that script somewhere along with your data, so that you not only knew what you did, but you could easily re-do it.
I've looked at distutils and it's not immediately obvious that it was designed to solve the distribution problem in the particular what I want to try to solve it, although I suppose I may be able to extend distutils with special commands to do something like this?
I want a way to say something like this:
Install my module here:
By default, place configuration data here:
But if the installer specifies different configuration options when running setup.py, then use those values when generating mypackage.conf from mypackage.conf.template.
Of course, the person installing the package should be able to override where configuration data is stored:
$ python setup.py --configdir=/var/foo
If such an option existed, its effect might be to generate a configuration file in the root of the package's module distribution folder:
containing something readable by ConfigParser:
[locations] prefix=/etc/mymodule/ connections=%prefix/connections.xml
So that at runtime, the module first looks to the configuration information in its package distribution to determine where it needs to look for the distribution-specific configuration information? Shwew.
Has anybody tackled this problem or do most folks just punt on this issue and assume they can plop configuration information alongside their distributed modules without providing any easy way for the person who installs it to override that?
-->"Mark" == Mark McEahern firstname.lastname@example.org writes:
Mark> Here's the short form of my question: How do people generally Mark> keep their Python modules and any configuration data for those Mark> Python modules separate?
i use the distutils/setup.py data_files option.
Mark> I'm looking for both conventions (e.g., config goes in Mark> /etc/foo) and mechanics (e.g., use distutils with options x, Mark> y, and z).
i put static config into $(PREFIX)/etc.
here's a cutting from my setup.py
<...> data_files = [("info", ["doc/pe4.info"]), ("doc", ["doc/pe4.pdf", "doc/pe4.html"]), ("etc", ["etc/pe4_client.pem", "etc/pe4_elvin_ca.pem"]), ] <...>
Mark> Install my module here: Mark> /usr/local/lib/pythonX.Y/site-packages/
Mark> By default, place configuration data here: Mark> /etc/my-package/
normally, configuration data for /usr/local packages would go into /usr/local/etc (ie. /usr/local is the install prefix).
Mark> Has anybody tackled this problem or do most folks just punt on Mark> this issue and assume they can plop configuration information Mark> alongside their distributed modules without providing any easy Mark> way for the person who installs it to override that?
it's a problem.
where you run into distutils limits like this, at least you have the option to sub-class the actions (i override the install class, for example).
the biggest problem is the lack of documentation -- you really have to just read the code.