On Tue, 2020-06-23 at 15:51 -0700, David Mathog wrote:
What I am after is some method of keeping exactly one copy of each package-version in the common area (ie, one might find foo-1.2, foo-1.7, and foo-2.3 there), while also presenting only the one version of each (let's say foo-1.7) to a particular installed program. On linux it might do that by making soft links to the common PYTHONPATH area from another directory for which it sets PYTHONPATH for the application. Finally, this has to be usable by any account which has read execute access to the main directory.
Does such a beast exist? If so, please point me to it!
zc.buildout and zc.recipe.egg can do something very much like this.
zc.buildout tries hard to maintain reproducibility and isolation, and one of the ways it
does this is by keeping each package in its own .egg directory. It then generates the
entry-point scripts and a REPL with
sys.path explicitly set to reference
exactly the versions specified. There's no virtualenv-like activation step that puts all
those scripts on the path, though, so one must either do that manually or just invoke the
generated scripts directly.
For example, here's a buildout configuration that specifies a shared directory to store eggs in. It also has some parts using zc.recipe.egg, one that will use zope.interface 4 and one that will use zope.interface 5 (egg specifications can be arbitrarily complex, of course, dependencies are followed, etc):
[buildout] eggs-directory = /<path>/buildout-eggs abi-tag-eggs = true parts = old-interface new-interface new-interface-plus
[old-interface] recipe = zc.recipe.egg eggs = zope.interface == 4.0 interpreter = old-py
[new-interface] recipe = zc.recipe.egg eggs = zope.interface == 5.1 interpreter = new-py
[new-interface-plus] recipe = zc.recipe.egg eggs = zope.interface == 5.1 zope.component interpreter = new-py-plus
buildout, I have three REPL files (if zope.interface had
defined any entry point scripts, I could also have had those generated):
$ head bin/new-py
sys.path[0:0] = [ '/<path>/buildout-eggs/pypy_73/zope.interface-5.1.0-py2.7.egg', '/<path/to/python>/site-packages', ]
$ head bin/old-py
sys.path[0:0] = [ '/<path>/buildout-eggs/pypy_73/zope.interface-4.0.0-py2.7.egg', '/<path/to/python>/site-packages', ]
$ head bin/new-py-plus
sys.path[0:0] = [ '/<path>/buildout-eggs/pypy_73/zope.interface-5.1.0-py2.7.egg', '/<path>/buildout-eggs/pypy_73/zope.component-4.6.1-py2.7.egg', '/<path/to/python>/site-packages', ]
I've got a collection of zope.interface eggs referenced from a variety of different buildouts (at least at one point in time) and from a variety of different Python implementations, but only ever one copy of each:
$ ls -ld buildout-eggs//zope.interface Permissions Size User Date Modified Name drwxr-xr-x - jmadden 2017-05-04 06:53 buildout-eggs/pypy_41/zope.interface-4.4.0-py2.7.egg/ drwxr-xr-x - jmadden 2017-05-04 07:00 buildout-eggs/cp27m/zope.interface-4.4.0-py2.7-macosx-10.12-x86_64.egg/ drwxr-xr-x - jmadden 2017-05-09 17:55 buildout-eggs/cp34m/zope.interface-4.4.0-py3.4-macosx-10.12-x86_64.egg/ drwxr-xr-x - jmadden 2017-06-08 10:20 buildout-eggs/cp27m/zope.interface-4.4.1-py2.7-macosx-10.12-x86_64.egg/ drwxr-xr-x - jmadden 2017-07-11 10:07 buildout-eggs/cp36m/zope.interface-4.4.2-py3.6-macosx-10.12-x86_64.egg/ drwxr-xr-x - jmadden 2017-12-08 11:10 buildout-eggs/cp27m/zope.interface-3.6.7-py2.7-macosx-10.13-x86_64.egg/ drwxr-xr-x - jmadden 2018-05-07 11:05 buildout-eggs/cp27m/zope.interface-4.1.3-py2.7-macosx-10.13-x86_64.egg/ drwxr-xr-x - jmadden 2020-03-13 07:53 buildout-eggs/cp27m/zope.interface-4.6.0-py2.7-macosx-10.15-x86_64.egg/ drwxr-xr-x - jmadden 2020-04-08 07:32 buildout-eggs/cp38/zope.interface-5.1.0-py3.8-macosx-10.15-x86_64.egg/ drwxr-xr-x - jmadden 2020-05-17 09:30 buildout-eggs/cp27m/zope.interface-5.1.0-py2.7.egg/ drwxr-xr-x - jmadden 2020-06-11 07:42 buildout-eggs/pypy_73/zope.interface-5.1.0-py2.7.egg/ drwxr-xr-x - jmadden 2020-06-25 11:58 buildout-eggs/pypy_73/zope.interface-4.0.0-py2.7.egg/