Buildout: How to include installed packages in sys.path?
Hello. I've written a Buildout recipe which relies on a function defined in a package whose name is set in the recipe's options. That package must have already been installed. According to the documentation, I should use zc.buildout.easy_install:install() like this: install(["mydistribution"], None) But if I use None, I get an exception because Buildout tries to use .startswith() with None. Then, if instead of None, I use buildout['buildout']['directory'], it tries to install the distribution pulling all its dependencies (ignoring the fact I set a download cache in ~/.buildout/default.cfg). I just want to make a package available in sys.path. How should I do that from a Buildout recipe? Note I really just want to add it sys.path, not get the actual function/module, because it's going to be used by a function called from my recipe, not by my recipe. The following code illustrates how the recipe will look like: ==== class MyRecipe(object): def __init__(self, buildout, name, options): self.the_package = options['package'] def install(self): add_to_sys_path(self.the_package) do_something_useful(self.the_package) return () ==== I'm using zc.buildout 1.4.3. Thanks in advance. -- Gustavo Narea. Software Developer. 2degrees, Ltd. <http://www.2degreesnetwork.com/>.
On Thu, Feb 11, 2010 at 12:37 PM, Gustavo Narea <gustavonarea@2degreesnetwork.com> wrote:
Hello.
I've written a Buildout recipe which relies on a function defined in a package whose name is set in the recipe's options. That package must have already been installed.
According to the documentation, I should use zc.buildout.easy_install:install() like this: install(["mydistribution"], None)
But if I use None, I get an exception because Buildout tries to use .startswith() with None. Then, if instead of None, I use buildout['buildout']['directory'], it tries to install the distribution pulling all its dependencies (ignoring the fact I set a download cache in ~/.buildout/default.cfg).
I just want to make a package available in sys.path. How should I do that from a Buildout recipe?
Note I really just want to add it sys.path, not get the actual function/module, because it's going to be used by a function called from my recipe, not by my recipe.
The short answer is that buildout doesn't support this. You can do it, but with a lot of work. As the documentation for install, http://pypi.python.org/pypi/zc.buildout#distribution-installation, says, if the second argument is None, it will fail unless the package is already installed. You need to specify the place to install to. It might be easier to use the zc.recipe.egg recipe from your recipe: http://pypi.python.org/pypi/zc.recipe.egg#egg-recipe-api-for-other-recipes After using that recipe to install the needed package, you'd need to use pkg_resources APIs to load the distribution into the running process. You might also look at the code buildout uses to load recipes. Jim -- Jim Fulton
Hello, Jim. Thank you very much for your response. I modified by recipe to use zc.recipe.egg like this: ==== from zc.recipe.egg import Scripts # ... egg = Scripts(buildout, name, options) egg.working_set((distribution_name,)) # ... do_something_useful(distribution_name) ==== But still do_something_useful() is failing (distribution_name is not in sys.path) and I think that's because I didn't use pkg_resources as you suggested. I tried to do it, and read the documentation and source code for pkg_resources, but I was not able to find the functionality I should use. I'll elaborate on what the recipe actually does, just in case: It loads options from an external INI file into the parts section, so other parts can use such variables. To be precise, it loads the variables from a PasteDeploy configuration file. Here's its code (it's just ~30 LOC): http://bitbucket.org/2degrees/deployrecipes/src/tip/deployrecipes.py And here's its documentation: http://packages.python.org/deployrecipes/ The tricky thing is, these files contain an option which is the string representation of a function. I don't need to load the actual function, all I want is to get all those options as strings. But PasteDeploy will always attempt to resolve that option and thus I must make it available. I hope it's not that hard to do with zc.recipe.eggs. Cheers, - Gustavo. On 11/02/10 22:48, Jim Fulton wrote:
The short answer is that buildout doesn't support this. You can do it, but with a lot of work. As the documentation for install, http://pypi.python.org/pypi/zc.buildout#distribution-installation, says, if the second argument is None, it will fail unless the package is already installed. You need to specify the place to install to.
It might be easier to use the zc.recipe.egg recipe from your recipe:
http://pypi.python.org/pypi/zc.recipe.egg#egg-recipe-api-for-other-recipes
After using that recipe to install the needed package, you'd need to use pkg_resources APIs to load the distribution into the running process.
You might also look at the code buildout uses to load recipes.
-- Gustavo Narea. Software Developer. 2degrees, Ltd. <http://www.2degreesnetwork.com/>.
On Fri, Feb 12, 2010 at 6:48 AM, Gustavo Narea <gustavonarea@2degreesnetwork.com> wrote:
Hello, Jim.
Thank you very much for your response.
I modified by recipe to use zc.recipe.egg like this: ==== from zc.recipe.egg import Scripts # ... egg = Scripts(buildout, name, options) egg.working_set((distribution_name,)) # ... do_something_useful(distribution_name) ====
But still do_something_useful() is failing (distribution_name is not in sys.path) and I think that's because I didn't use pkg_resources as you suggested. I tried to do it, and read the documentation and source code for pkg_resources, but I was not able to find the functionality I should use.
Change your call to working_set to capture the working set: requirements, ws = egg.working_set((distribution_name,)) Then: for dist in ws: pkg_resources.working_set.add(dist) Jim -- Jim Fulton
Thank you, Jim. It worked! - Gustavo. On 12/02/10 22:49, Jim Fulton wrote:
On Fri, Feb 12, 2010 at 6:48 AM, Gustavo Narea <gustavonarea@2degreesnetwork.com> wrote:
Hello, Jim.
Thank you very much for your response.
I modified by recipe to use zc.recipe.egg like this: ==== from zc.recipe.egg import Scripts # ... egg = Scripts(buildout, name, options) egg.working_set((distribution_name,)) # ... do_something_useful(distribution_name) ====
But still do_something_useful() is failing (distribution_name is not in sys.path) and I think that's because I didn't use pkg_resources as you suggested. I tried to do it, and read the documentation and source code for pkg_resources, but I was not able to find the functionality I should use.
Change your call to working_set to capture the working set:
requirements, ws = egg.working_set((distribution_name,))
Then:
for dist in ws: pkg_resources.working_set.add(dist)
Jim
-- Jim Fulton
-- Gustavo Narea. Software Developer. 2degrees, Ltd. <http://www.2degreesnetwork.com/>.
participants (2)
-
Gustavo Narea -
Jim Fulton