I have devised the following hack which works for the time being. I add the following line to two new files called 000-fix-easyinstall.pth and zzz-fix-easyinstall.pth:
import sys; sys.__egginsert = len(sys.path)
This causes the next path re-arrangement by an easy-install.pth to be after all current paths.
Why two:
The .pth are sorted by site.py. So 000-fix-easyinstall.pth will get called first and set __egginsert after any current paths (such as the core paths and the user site-packages path). This way, when the user easy-install.pth is processed if there is one, any path re-arrangements will occur after them.
Now the problem with just one is, if there are any other user .pth files that get processed after the user easy-install.pth, they will add paths but not update the egginsert value. So when the system easy-install.pth gets invoked, those paths will be moved above the user paths that were added after the user easy-install.pth. The solution to this is the second fix file zzz-fix-easyinstall.pth. Since it is processed last, i ensures that the egginsert is after any added paths.
To see it in action:
import sys for i in sys.path: print(i)
Before:
/home/allen/.local/lib/python3.4/site-packages/setuptools /usr/local/lib/python3.4/dist-packages/psutil-3.4.2-py3.4-linux-x86_64.egg /usr/local/lib/python3.4/dist-packages/docker_py-1.7.0_rc2-py3.4.egg /usr/local/lib/python3.4/dist-packages/raven-5.10.1-py3.4.egg /usr/local/lib/python3.4/dist-packages/Jinja2-2.8-py3.4.egg /usr/local/lib/python3.4/dist-packages/aiohttp-0.20.2-py3.4-linux-x86_64.egg /usr/local/lib/python3.4/dist-packages/jsonschema-2.5.1-py3.4.egg /usr/local/lib/python3.4/dist-packages/websocket_client-0.35.0-py3.4.egg /usr/local/lib/python3.4/dist-packages/requests-2.9.1-py3.4.egg /usr/local/lib/python3.4/dist-packages/MarkupSafe-0.23-py3.4-linux-x86_64.egg /usr/lib/python3/dist-packages /usr/local/lib/python3.4/dist-packages/paramiko-1.16.0-py3.4.egg /usr/local/lib/python3.4/dist-packages/configobj-5.0.6-py3.4.egg /usr/local/lib/python3.4/dist-packages/ecdsa-0.13-py3.4.egg /usr/local/lib/python3.4/dist-packages/pycrypto-2.6.1-py3.4-linux-x86_64.egg /usr/local/lib/python3.4/dist-packages/gns3_server-1.4.6-py3.4.egg /usr/local/lib/python3.4/dist-packages/gns3_gui-1.4.6-py3.4.egg /usr/local/lib/python3.4/dist-packages/gns3_net_converter-1.3.0-py3.4.egg /usr/lib/python3.4 /usr/lib/python3.4/plat-x86_64-linux-gnu /usr/lib/python3.4/lib-dynload /home/allen/.local/lib/python3.4/site-packages /usr/local/lib/python3.4/dist-packages
After:
/usr/lib/python3.4 /usr/lib/python3.4/plat-x86_64-linux-gnu /usr/lib/python3.4/lib-dynload /home/allen/.local/lib/python3.4/site-packages /home/allen/.local/lib/python3.4/site-packages/setuptools /usr/local/lib/python3.4/dist-packages/psutil-3.4.2-py3.4-linux-x86_64.egg /usr/local/lib/python3.4/dist-packages/docker_py-1.7.0_rc2-py3.4.egg /usr/local/lib/python3.4/dist-packages/raven-5.10.1-py3.4.egg /usr/local/lib/python3.4/dist-packages/Jinja2-2.8-py3.4.egg /usr/local/lib/python3.4/dist-packages/aiohttp-0.20.2-py3.4-linux-x86_64.egg /usr/local/lib/python3.4/dist-packages/jsonschema-2.5.1-py3.4.egg /usr/local/lib/python3.4/dist-packages/websocket_client-0.35.0-py3.4.egg /usr/local/lib/python3.4/dist-packages/requests-2.9.1-py3.4.egg /usr/local/lib/python3.4/dist-packages/MarkupSafe-0.23-py3.4-linux-x86_64.egg /usr/lib/python3/dist-packages /usr/local/lib/python3.4/dist-packages/paramiko-1.16.0-py3.4.egg /usr/local/lib/python3.4/dist-packages/configobj-5.0.6-py3.4.egg /usr/local/lib/python3.4/dist-packages/ecdsa-0.13-py3.4.egg /usr/local/lib/python3.4/dist-packages/pycrypto-2.6.1-py3.4-linux-x86_64.egg /usr/local/lib/python3.4/dist-packages/gns3_server-1.4.6-py3.4.egg /usr/local/lib/python3.4/dist-packages/gns3_gui-1.4.6-py3.4.egg /usr/local/lib/python3.4/dist-packages/gns3_net_converter-1.3.0-py3.4.egg /usr/local/lib/python3.4/dist-packages
This keeps the core paths first, then the user paths, then the system paths
Brian Allen Vanderburg II
On 07/04/2016 04:59 AM, Piotr Dobrogost wrote:
On Sun, Jul 3, 2016 at 7:05 PM, Brian Allen Vanderburg II via Distutils-SIG distutils-sig@python.org wrote:
The issue I'm having is when I install a test/local version of a package in my user site-packages, but the same package is also installed in a system path and has a .pth entry in the system easy-install.pth. It seems that from a path perspective, items under the user path should automatically take precedence over the system path. However, because of the way the paths are rearranged in easy-install.pth, the system path always takes precedence since the easy-install.pth line moves all added items to the front of the path.
You might want to take a look at issue "pip list reports wrong version of package installed both in system site and user site" at https://github.com/pypa/pip/issues/1018. One "solution" is not to install any packages in virtualenv with setuptools (which hacks easy-install.pth file as you described). Install only with pip instead.
Regards, Piotr Dobrogost
So while it seems like the path should be something like:
cwd:core-paths:user-site-paths:user-pth-paths:system-dist-paths:system-pth-paths
The path turns out to be:
cwd:user-pth-paths:system-pth-paths:core-paths:user-site-paths:system-dist-paths
Is there a reason why the .pth files even contain the final line that rearranges the paths for all the .eggs to be at the start of the system path?
For the time being, my work-around is to manually add a .pth file to my user site-packages path for each item in site-packages as well, and add the lines needed so that when the system easy-install.pth is processed, it doesn't move all:
import sys; sys.__plen = len(sys.path)
<my custom entries for each item in user site packages> import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)
All of that is required, because without it, sys.__egginsert will still be unset, and when site.py processes the system path, items in easy-install.pth will be moved above any user .pth items as well.
However, ideally it seems like this path should automatically have precedence over system packages, and it would if not for this line in the various easy-install.pth files:
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)
I guess I just find myself wondering what is the purpose of the above line in easy-install.pth files (for python3, but apparently not python2), and why not just keep the path in the order that site.py would add them.
Thanks,
Brian Allen Vanderburg II
Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig