Re: package management - common storage while keeping the versions straight
On Tue, 2020-06-23 at 15:51 -0700, David Mathog wrote:
What I am after is some method of keeping exactly one copy of each package-version in the common area (ie, one might find foo-1.2, foo-1.7, and foo-2.3 there), while also presenting only the one version of each (let's say foo-1.7) to a particular installed program. On linux it might do that by making soft links to the common PYTHONPATH area from another directory for which it sets PYTHONPATH for the application. Finally, this has to be usable by any account which has read execute access to the main directory.
Does such a beast exist? If so, please point me to it!
zc.buildout[1] and zc.recipe.egg[2] can do something very much like this. zc.buildout tries hard to maintain reproducibility and isolation, and one of the ways it does this is by keeping each package in its own .egg directory. It then generates the entry-point scripts and a REPL with ``sys.path`` explicitly set to reference exactly the versions specified. There's no virtualenv-like activation step that puts all those scripts on the path, though, so one must either do that manually or just invoke the generated scripts directly. For example, here's a buildout configuration that specifies a shared directory to store eggs in. It also has some parts using zc.recipe.egg, one that will use zope.interface 4 and one that will use zope.interface 5 (egg specifications can be arbitrarily complex, of course, dependencies are followed, etc): [buildout] eggs-directory = /<path>/buildout-eggs abi-tag-eggs = true parts = old-interface new-interface new-interface-plus [old-interface] recipe = zc.recipe.egg eggs = zope.interface == 4.0 interpreter = old-py [new-interface] recipe = zc.recipe.egg eggs = zope.interface == 5.1 interpreter = new-py [new-interface-plus] recipe = zc.recipe.egg eggs = zope.interface == 5.1 zope.component interpreter = new-py-plus After running `buildout`, I have three REPL files (if zope.interface had defined any entry point scripts, I could also have had those generated): $ head bin/new-py #!<path/to/python> import sys sys.path[0:0] = [ '/<path>/buildout-eggs/pypy_73/zope.interface-5.1.0-py2.7.egg', '/<path/to/python>/site-packages', ] $ head bin/old-py #!<path/to/python> import sys sys.path[0:0] = [ '/<path>/buildout-eggs/pypy_73/zope.interface-4.0.0-py2.7.egg', '/<path/to/python>/site-packages', ] $ head bin/new-py-plus #!<path/to/python> import sys sys.path[0:0] = [ '/<path>/buildout-eggs/pypy_73/zope.interface-5.1.0-py2.7.egg', '/<path>/buildout-eggs/pypy_73/zope.component-4.6.1-py2.7.egg', '/<path/to/python>/site-packages', ] I've got a collection of zope.interface eggs referenced from a variety of different buildouts (at least at one point in time) and from a variety of different Python implementations, but only ever one copy of each: $ ls -ld buildout-eggs/*/zope.interface* Permissions Size User Date Modified Name drwxr-xr-x - jmadden 2017-05-04 06:53 buildout-eggs/pypy_41/zope.interface-4.4.0-py2.7.egg/ drwxr-xr-x - jmadden 2017-05-04 07:00 buildout-eggs/cp27m/zope.interface-4.4.0-py2.7-macosx-10.12-x86_64.egg/ drwxr-xr-x - jmadden 2017-05-09 17:55 buildout-eggs/cp34m/zope.interface-4.4.0-py3.4-macosx-10.12-x86_64.egg/ drwxr-xr-x - jmadden 2017-06-08 10:20 buildout-eggs/cp27m/zope.interface-4.4.1-py2.7-macosx-10.12-x86_64.egg/ drwxr-xr-x - jmadden 2017-07-11 10:07 buildout-eggs/cp36m/zope.interface-4.4.2-py3.6-macosx-10.12-x86_64.egg/ drwxr-xr-x - jmadden 2017-12-08 11:10 buildout-eggs/cp27m/zope.interface-3.6.7-py2.7-macosx-10.13-x86_64.egg/ drwxr-xr-x - jmadden 2018-05-07 11:05 buildout-eggs/cp27m/zope.interface-4.1.3-py2.7-macosx-10.13-x86_64.egg/ drwxr-xr-x - jmadden 2020-03-13 07:53 buildout-eggs/cp27m/zope.interface-4.6.0-py2.7-macosx-10.15-x86_64.egg/ drwxr-xr-x - jmadden 2020-04-08 07:32 buildout-eggs/cp38/zope.interface-5.1.0-py3.8-macosx-10.15-x86_64.egg/ drwxr-xr-x - jmadden 2020-05-17 09:30 buildout-eggs/cp27m/zope.interface-5.1.0-py2.7.egg/ drwxr-xr-x - jmadden 2020-06-11 07:42 buildout-eggs/pypy_73/zope.interface-5.1.0-py2.7.egg/ drwxr-xr-x - jmadden 2020-06-25 11:58 buildout-eggs/pypy_73/zope.interface-4.0.0-py2.7.egg/ ~Jason [1] http://www.buildout.org/ ; I recommend version 3, which uses pip to find eggs [2] https://pypi.org/project/zc.recipe.egg/
participants (1)
-
Jason Madden