Files under /etc and /usr/share

Hi, It is some weeks I put easy-deb on http://easy-deb.sf.net. I'm now considering improvements to make a new release. I have two things in mind: 1) make easy-deb a setup command (now it is an external tool) 2) provide a way to put files into /etc About point 2): Is it possible to install things into /etc using setuptools/easy_install? I think that this feature will be very appreciated. I know setup.py philosopy is to put everything inside eggs, but config files are a really special case! It would be nice beeing able to tell in setup.py what the config files/dirs are. The generated egg should install the config files on user request (using a gen_config script) or just when the neded are not config files on the system. Another idea is for heavy data. Consider for example moin_moin. Using easy-deb i managed to aotomatically pacakge it into an egg (which is contained into a deb-package). The problem here is that it installs the template files inside the egg. Since template files must be copied "somewere" it is bat to put them into the egg where they are hidden/zipped. Moreover it would be nice to make the eggs inside /usr/lib/python?.?/site-packages more lightweight removing package data from them. There are 2 kind of package data: data that is used *by* the module code and data that is used *with* the module. moinmoin is just an example of module whose data is not used by the code but with the code. I have no idea how this could be done, but it would be good (IMHO). Another nice thing to have in setuptools is a --nodeps option. Packagers really need to install an egg (into a package of course) even if its dependencies are not on the system. Skipped dependencies could be reported inside a skipped_deps.txt file or on stdout (so that packagers can add the dependency info to the generated packages). Ciao Vincenzo ___________________________________ Yahoo! Messenger: chiamate gratuite in tutto il mondo http://it.messenger.yahoo.com

At 12:38 PM 9/18/2005 +0200, Vincenzo Di Massa wrote:
Is it possible to install things into /etc using setuptools/easy_install? I think that this feature will be very appreciated. I know setup.py philosopy is to put everything inside eggs, but config files are a really special case! It would be nice beeing able to tell in setup.py what the config files/dirs are. The generated egg should install the config files on user request (using a gen_config script) or just when the neded are not config files on the system.
I think that this is very application-specific as to how it should be handled. It's very easy to create a package that contains configuration files, with a function in __init__.py to install them, perhaps with some options or an interactive setup facility. Listing the function in setup.py then allows a script wrapper to be created for setting up the data. Using pkg_resources.resource_filename() with a directory resource will unpack an entire directory tree to a temporary location, or give you the location of files that are already unpacked, so you can then use shutil or other normal ways of copying files.
Another idea is for heavy data. Consider for example moin_moin. Using easy-deb i managed to aotomatically pacakge it into an egg (which is contained into a deb-package). The problem here is that it installs the template files inside the egg. Since template files must be copied "somewere" it is bat to put them into the egg where they are hidden/zipped. Moreover it would be nice to make the eggs inside /usr/lib/python?.?/site-packages more lightweight removing package data from them.
I don't really agree. Projects with this kind of data are best off if they provide a custom installation tool. Well, I should say that their *users* are best off if they provide an installation tool, rather than dumping the files in one place. Further, it is easier to support global installation of a package for multiple users, or for a single user with multiple installations of the tool. (E.g. multiple wikis.) If the data installation is done by easy_install, it can only be done once, and without any real options as to how it's done. If it's done by a function included with the package, then it can be done as many times as desired, and with whatever options the project wants to provide the user with.
There are 2 kind of package data: data that is used *by* the module code and data that is used *with* the module. moinmoin is just an example of module whose data is not used by the code but with the code.
Well, in my way of thinking, such templates *are* used by the code, it's just that they should be used by the project's "instance install/config" script(s).
Another nice thing to have in setuptools is a --nodeps option. Packagers really need to install an egg (into a package of course) even if its dependencies are not on the system. Skipped dependencies could be reported inside a skipped_deps.txt file or on stdout (so that packagers can add the dependency info to the generated packages).
We could have a --record-deps option, but you can also just pull the dependencies from the Distribution.requires() method. I have had it on my list to have a --no-deps option, though, I just haven't gotten around to it yet.

Alle 17:49, domenica 18 settembre 2005, hai scritto:
At 12:38 PM 9/18/2005 +0200, Vincenzo Di Massa wrote:
Is it possible to install things into /etc using setuptools/easy_install? I think that this feature will be very appreciated. I know setup.py philosopy is to put everything inside eggs, but config files are a really special case! It would be nice beeing able to tell in setup.py what the config files/dirs are. The generated egg should install the config files on user request (using a gen_config script) or just when the neded are not config files on the system.
I think that this is very application-specific as to how it should be handled. It's very easy to create a package that contains configuration files, with a function in __init__.py to install them, perhaps with some options or an interactive setup facility. Listing the function in setup.py then allows a script wrapper to be created for setting up the data.
Using pkg_resources.resource_filename() with a directory resource will unpack an entire directory tree to a temporary location, or give you the location of files that are already unpacked, so you can then use shutil or other normal ways of copying files.
What you say seems to be the right thing (tm) :-). What I am proposing is nothing more that a convention to do config files installation. Maybe it is just something that can be explained into the documentation. It would be nice to ask setup.py writers to adopt the "best practice" that you descibed before. Maybe we can define what the name of the function is and what parameters it requires: this will make easy and obvious what is the right way of handling config options. Once the function name is standardized... why not provide a default implementation that automatically installs the files you specify somewhere? It is tedious to have to write code to just handle a little set of config files. It is a very common use case, so it would be good to provide some authomatism to: 1) make the distribution of many modules easier 2) provide a standard way to do one thing
Another idea is for heavy data. Consider for example moin_moin. Using easy-deb i managed to aotomatically pacakge it into an egg (which is contained into a deb-package). The problem here is that it installs the template files inside the egg. Since template files must be copied "somewere" it is bat to put them into the egg where they are hidden/zipped. Moreover it would be nice to make the eggs inside /usr/lib/python?.?/site-packages more lightweight removing package data from them.
I don't really agree. Projects with this kind of data are best off if they provide a custom installation tool. Well, I should say that their *users* are best off if they provide an installation tool, rather than dumping the files in one place. Further, it is easier to support global installation of a package for multiple users, or for a single user with multiple installations of the tool. (E.g. multiple wikis.) If the data installation is done by easy_install, it can only be done once, and without any real options as to how it's done. If it's done by a function included with the package, then it can be done as many times as desired, and with whatever options the project wants to provide the user with.
I did not explain myself very well, sorry. What you say is absolutely right! What I meant is: Suppose you have a module that has a an installation tool that is able to install data files/templates to multiple dirs at different times. All the copies are taken from somewhere. Now the source copy is inside the egg. My proposal was to move the source copy out of the egg. The reasons are: 1) you should be able to modify the default template 2) the template can be big an big templates belong to something different from /usr/lib/python?.?/... 3) provide a standard way to do one thing Maybe setuptools could optionally put a couple scripts inside generated eggs that can: 1) put default config files in place 2) move the templates somewhere else (like /usr/share) This way config files and data files could be handled automatically like setuptools does for dependencies: before executing the module could call pkg_resurces.config_files("modulename") and pkg_resources.data_files("modulename") to make sure that the config files exist and that data files are where they shuold be.
There are 2 kind of package data: data that is used *by* the module code and data that is used *with* the module. moinmoin is just an example of module whose data is not used by the code but with the code.
Well, in my way of thinking, such templates *are* used by the code, it's just that they should be used by the project's "instance install/config" script(s).
I understand your point of view, any template can be seen as used by the module's code, but... There is a little difference: there is data that the user will hand edit, and data used an modified only by the module. There is data that must installed only once, and data (like moinmoin templates) that can be installed multiple times. I wanted to distinguish data that can be somehow "hidden" from data that must be "visible".
Another nice thing to have in setuptools is a --nodeps option. Packagers really need to install an egg (into a package of course) even if its dependencies are not on the system. Skipped dependencies could be reported inside a skipped_deps.txt file or on stdout (so that packagers can add the dependency info to the generated packages).
We could have a --record-deps option, but you can also just pull the dependencies from the Distribution.requires() method. I have had it on my list to have a --no-deps option, though, I just haven't gotten around to it yet.
You are doing a really good job with setuptools! I really like how they are now. I'm just trying to add my 0.02€ :-) I hope I'm not adding -2€ Ciao Vincenzo ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it

Sorry for the previous post... The yhaoo antispam system added bul_k to the subject... Alle 17:49, domenica 18 settembre 2005, hai scritto:
At 12:38 PM 9/18/2005 +0200, Vincenzo Di Massa wrote:
Is it possible to install things into /etc using setuptools/easy_install? I think that this feature will be very appreciated. I know setup.py philosopy is to put everything inside eggs, but config files are a really special case! It would be nice beeing able to tell in setup.py what the config files/dirs are. The generated egg should install the config files on user request (using a gen_config script) or just when the neded are not config files on the system.
I think that this is very application-specific as to how it should be handled. It's very easy to create a package that contains configuration files, with a function in __init__.py to install them, perhaps with some options or an interactive setup facility. Listing the function in setup.py then allows a script wrapper to be created for setting up the data.
Using pkg_resources.resource_filename() with a directory resource will unpack an entire directory tree to a temporary location, or give you the location of files that are already unpacked, so you can then use shutil or other normal ways of copying files.
What you say seems to be the right thing (tm) :-). What I am proposing is nothing more that a convention to do config files installation. Maybe it is just something that can be explained into the documentation. It would be nice to ask setup.py writers to adopt the "best practice" that you descibed before. Maybe we can define what the name of the function is and what parameters it requires: this will make easy and obvious what is the right way of handling config options. Once the function name is standardized... why not provide a default implementation that automatically installs the files you specify somewhere? It is tedious to have to write code to just handle a little set of config files. It is a very common use case, so it would be good to provide some authomatism to: 1) make the distribution of many modules easier 2) provide a standard way to do one thing
Another idea is for heavy data. Consider for example moin_moin. Using easy-deb i managed to aotomatically pacakge it into an egg (which is contained into a deb-package). The problem here is that it installs the template files inside the egg. Since template files must be copied "somewere" it is bat to put them into the egg where they are hidden/zipped. Moreover it would be nice to make the eggs inside /usr/lib/python?.?/site-packages more lightweight removing package data from them.
I don't really agree. Projects with this kind of data are best off if they provide a custom installation tool. Well, I should say that their *users* are best off if they provide an installation tool, rather than dumping the files in one place. Further, it is easier to support global installation of a package for multiple users, or for a single user with multiple installations of the tool. (E.g. multiple wikis.) If the data installation is done by easy_install, it can only be done once, and without any real options as to how it's done. If it's done by a function included with the package, then it can be done as many times as desired, and with whatever options the project wants to provide the user with.
I did not explain myself very well, sorry. What you say is absolutely right! What I meant is: Suppose you have a module that has a an installation tool that is able to install data files/templates to multiple dirs at different times. All the copies are taken from somewhere. Now the source copy is inside the egg. My proposal was to move the source copy out of the egg. The reasons are: 1) you should be able to modify the default template 2) the template can be big an big templates belong to something different from /usr/lib/python?.?/... 3) provide a standard way to do one thing Maybe setuptools could optionally put a couple scripts inside generated eggs that can: 1) put default config files in place 2) move the templates somewhere else (like /usr/share) This way config files and data files could be handled automatically like setuptools does for dependencies: before executing the module could call pkg_resurces.config_files("modulename") and pkg_resources.data_files("modulename") to make sure that the config files exist and that data files are where they shuold be.
There are 2 kind of package data: data that is used *by* the module code and data that is used *with* the module. moinmoin is just an example of module whose data is not used by the code but with the code.
Well, in my way of thinking, such templates *are* used by the code, it's just that they should be used by the project's "instance install/config" script(s).
I understand your point of view, any template can be seen as used by the module's code, but... There is a little difference: there is data that the user will hand edit, and data used an modified only by the module. There is data that must installed only once, and data (like moinmoin templates) that can be installed multiple times. I wanted to distinguish data that can be somehow "hidden" from data that must be "visible".
Another nice thing to have in setuptools is a --nodeps option. Packagers really need to install an egg (into a package of course) even if its dependencies are not on the system. Skipped dependencies could be reported inside a skipped_deps.txt file or on stdout (so that packagers can add the dependency info to the generated packages).
We could have a --record-deps option, but you can also just pull the dependencies from the Distribution.requires() method. I have had it on my list to have a --no-deps option, though, I just haven't gotten around to it yet.
You are doing a really good job with setuptools! I really like how they are now. I'm just trying to add my 0.02€ :-) I hope I'm not adding -2€ Ciao Vincenzo ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it

I haven't followed this discussion in detail, but I think I've started doing something like what Vincenzo asked about. It's not very general, though. In my case, I just put a file paste_deploy_config.ini_tmpl into the .egg-info directory, and then use that when deploying a web application. The file itself is a template for configuration, and those variables are filled in as part of the deployment. A more general and maybe even simpler method would simply be an entry point, like [etc_files], which would be an object with certain methods, which could be used to install it in some location. I imagine such an object would indicate the variables needed to install it, default location for the configuration file(s), and possibly other post-install routines. Anyway, I think that's probably what you are getting at here...?
participants (3)
-
Ian Bicking
-
Phillip J. Eby
-
Vincenzo Di Massa