Just thought I'd pass this along, since it took me a little while to figure it out. Maybe this can go in docs somewhere, but I'm not sure where. So, I'm writing some code that uses egg plugins, and thus I need testing eggs. These need to be path-independent (since the checkouts might live anywhere), with no setup commands (you shouldn't have to install the testing version of the eggs to run the tests), and the eggs should be available only when you are running the tests (no global installation). I'm using py.test, so I add a conftest.py file which is loaded before any tests are imported. It's important all this is done before pkg_resources is imported (maybe there are methods in pkg_resources that can fix things after it was imported, but pkg_resources uses sys.path when it is imported, so if you adjust the path later then pkg_resources won't notice it). Anyway, here's the code I use: import os import sys import glob here = os.path.dirname(__file__) base = os.path.dirname(here) fake_packages = os.path.join(here, 'fake_packages') for egg_info_dir in glob.glob('%s/*/*.egg-info' % fake_packages): sys.path.append(os.path.dirname(egg_info_dir)) At first I tried adding fake_packages to sys.path; didn't work at all. But if I do site.addsitedir(fake_packages) then it would work. But this requires an .egg-link file in fake_packages, and that file has to have an absolute path (it can't be relative), but fake_packages could be anywhere. So in the end, I just need to add all the necessary paths; this means I can't test the case when --multi-version is used to install an egg, but I guess I won't worry about that. If you are curious about the base setup, I've checked in a minimal file layout of the whole thing into http://svn.pythonpaste.org/Paste/Deploy/trunk/ -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
At 06:00 PM 8/19/2005 -0500, Ian Bicking wrote:
Just thought I'd pass this along, since it took me a little while to figure it out. Maybe this can go in docs somewhere, but I'm not sure where.
So, I'm writing some code that uses egg plugins, and thus I need testing eggs. These need to be path-independent (since the checkouts might live anywhere), with no setup commands (you shouldn't have to install the testing version of the eggs to run the tests), and the eggs should be available only when you are running the tests (no global installation).
I'm kind of confused. Maybe what you want is to create dummy Distribution objects and add them to a WorkingSet, then use the WorkingSet's APIs in your tests (such as iter_entry_points, require(), etc.). It's not entirely clear to me what it is that you're testing, so I'm not sure. In setuptools.tests.test_resources there's a simple Metadata class that's used to create dummy distributions that nonetheless have metadata; you could use something like that to create a dummy Distribution. A WorkingSet is a collection of active Distributions, and it has methods like require(), iter_entry_points(), and so on. There is a default global WorkingSet that's used for pkg_resources API, but you can create your own instances for testing purposes, and manipulating it won't affect sys.path. The default working set has a listener subscribed to it that activates distributions on sys.path, but if you just create a WorkingSet for yourself it won't have this link. The "api_tests.txt" file demonstrates some WorkingSet APIs, and "pkg_resources.txt" has complete reference docs for its methods.
At first I tried adding fake_packages to sys.path; didn't work at all.
Because the subdirectories' names don't end in .egg; if they did and you just used EGG-INFO instead of .egg-info, it would work.
But if I do site.addsitedir(fake_packages) then it would work. But this requires an .egg-link file in fake_packages, and that file has to have an absolute path (it can't be relative), but fake_packages could be anywhere. So in the end, I just need to add all the necessary paths; this means I can't test the case when --multi-version is used to install an egg, but I guess I won't worry about that.
If you are curious about the base setup, I've checked in a minimal file layout of the whole thing into http://svn.pythonpaste.org/Paste/Deploy/trunk/
From what little I can tell about what you're trying to do, I think you should just create dummy Distribution objects and add them a test-only WorkingSet, and not bother having anything in the filesystem, unless you're testing code that also does stuff with the filesystem. In which case, you could just use pkg_resources' code to find and introspect packages, though, so as I said, I'm not sure what it is exactly you're testing. :)
Phillip J. Eby wrote:
At 06:00 PM 8/19/2005 -0500, Ian Bicking wrote:
Just thought I'd pass this along, since it took me a little while to figure it out. Maybe this can go in docs somewhere, but I'm not sure where.
So, I'm writing some code that uses egg plugins, and thus I need testing eggs. These need to be path-independent (since the checkouts might live anywhere), with no setup commands (you shouldn't have to install the testing version of the eggs to run the tests), and the eggs should be available only when you are running the tests (no global installation).
I'm kind of confused. Maybe what you want is to create dummy Distribution objects and add them to a WorkingSet, then use the WorkingSet's APIs in your tests (such as iter_entry_points, require(), etc.). It's not entirely clear to me what it is that you're testing, so I'm not sure.
I'm testing loading WSGI apps out of eggs using entry points. So there will be real code in those test packages, though just stubby code. I'm not testing setuptools per se, so dummy Distributions probably won't be of much use.
At first I tried adding fake_packages to sys.path; didn't work at all.
Because the subdirectories' names don't end in .egg; if they did and you just used EGG-INFO instead of .egg-info, it would work.
OK, I'll try that. I was surprised, though, that FakeApp.egg-link didn't work until I added the directory as addsitedir. Though I was flailing about to get it working, so I don't know what exactly combination of things I ended up using. I also realize it would be nice to refresh the egg_info before running the tests, but I'm not sure how well that will work if it happens in the same process as the tests themselves. Or how I run a setup() command from Python. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
Ian Bicking wrote:
At first I tried adding fake_packages to sys.path; didn't work at all.
Because the subdirectories' names don't end in .egg; if they did and you just used EGG-INFO instead of .egg-info, it would work.
OK, I'll try that.
That worked; as long as fake_packages is in the path (before I import pkg_resources), and if I rename FakeApp to FakeApp.egg, and symlink EGG-INFO to FakeApp.egg-info, then it works. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
At 10:04 PM 8/19/2005 -0500, Ian Bicking wrote:
Phillip J. Eby wrote:
I'm kind of confused. Maybe what you want is to create dummy Distribution objects and add them to a WorkingSet, then use the WorkingSet's APIs in your tests (such as iter_entry_points, require(), etc.). It's not entirely clear to me what it is that you're testing, so I'm not sure.
I'm testing loading WSGI apps out of eggs using entry points. So there will be real code in those test packages, though just stubby code. I'm not testing setuptools per se, so dummy Distributions probably won't be of much use.
Note that entry points don't have to point into the egg; they just get imported. Thus, you might as well point the entry points back into your test code, with no need to create a separate package. All you really need are Distribution objects with dummy "entry_points.txt" metadata resources.
At first I tried adding fake_packages to sys.path; didn't work at all.
Because the subdirectories' names don't end in .egg; if they did and you just used EGG-INFO instead of .egg-info, it would work.
OK, I'll try that. I was surprised, though, that FakeApp.egg-link didn't work until I added the directory as addsitedir. Though I was flailing about to get it working, so I don't know what exactly combination of things I ended up using.
I also realize it would be nice to refresh the egg_info before running the tests, but I'm not sure how well that will work if it happens in the same process as the tests themselves. Or how I run a setup() command from Python.
That's why I think you'll have an easier time just creating dummy Distributions with entry point metadata that links back into your test code; then you don't have to even have real packages inside the eggs, nor do you have to manipulate sys.path any. Since you're not testing setuptools, that should work fine for your purposes, if I understand them correctly. All you really want to test is that your code makes the right calls on a working set and invokes your stub entry points. Just make your code take a WorkingSet as a parameter, and default to pkg_resources.working_set (the global WorkingSet). Your test code will pass in its own WorkingSet instances, which will not affect sys.path or the global pkg_resources state. But your code will be able to call require(), resolve(), iter_entry_points(), and all the other standard APIs, because these are WorkingSet methods anyway. The global working_set's methods are exported by pkg_resources as module-level APIs, but code like yours should operate on a WorkingSet instance and just default to the global one.
participants (2)
-
Ian Bicking
-
Phillip J. Eby