Getting source from __file__ in egg?
If I have a package installed as a zip file egg and ask for its __file__ I might get a reference which looks like a normal file system path but isn't: % python Python 2.4.5 (#4, Apr 12 2008, 09:09:16) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import magnitude >>> magnitude.__file__ '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/magnitude-0.9.3-py2.4.egg/magnitude.pyc' >>> magnitude.__file__[:-1] '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/magnitude-0.9.3-py2.4.egg/magnitude.py' >>> import os >>> os.path.exists(magnitude.__file__[:-1]) False How do I actually lay my hands on the .py file source for the magnitude module in this case? Do I have to recognize somehow that .../magnitude-0.9.3-py2.4.egg is a zip archive and resort to extracting the Python source with the zipfile module? Thx, Skip Montanaro
Try pkg_resources.resource_string or pkg_resources.resource_file - it allows you to retrieve any object in an egg as a string or file-like object. :-) -----Original Message----- From: distutils-sig-bounces+fadhley.salim=uk.calyon.com@python.org [mailto:distutils-sig-bounces+fadhley.salim=uk.calyon.com@python.org] On Behalf Of skip@pobox.com Sent: 10 September 2008 12:14 To: distutils-sig@python.org Subject: [Distutils] Getting source from __file__ in egg? If I have a package installed as a zip file egg and ask for its __file__ I might get a reference which looks like a normal file system path but isn't: % python Python 2.4.5 (#4, Apr 12 2008, 09:09:16) [GCC 3.4.1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import magnitude >>> magnitude.__file__ '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/magnitude-0.9.3 -py2.4.egg/magnitude.pyc' >>> magnitude.__file__[:-1] '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/magnitude-0.9.3 -py2.4.egg/magnitude.py' >>> import os >>> os.path.exists(magnitude.__file__[:-1]) False How do I actually lay my hands on the .py file source for the magnitude module in this case? Do I have to recognize somehow that .../magnitude-0.9.3-py2.4.egg is a zip archive and resort to extracting the Python source with the zipfile module? Thx, Skip Montanaro _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig This email does not create a legal relationship between any member of the Crédit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Commission Bancaire in France and regulated by the Financial Services Authority for the conduct of business in the United Kingdom. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA.
On Sep 10, 2008, at 5:13 AM, skip@pobox.com wrote:
If I have a package installed as a zip file egg and ask for its __file__ I might get a reference which looks like a normal file system path but isn't:
I advise people to add to their distutils config file: [easy_install] zip_ok=False In addition, I suggest that setuptools should be changed to install eggs unzipped by default, as there is a list of problems and inconveniences with zipping eggs and, as far as I am aware, no measured benefits. Here is the issue ticket, in which your problem, Skip, is the most recent entry in the list: http://bugs.python.org/setuptools/issue33 Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month
Skip's issue isn't going to be resolved by avoiding zipfiles in the easy_install case. Really, none of the other issues you've mentioned in that ticket will be either. Zipfiles are a useful installation target for drop-and-go plugin installation, and they're useful for py2exe, py2app, etc. A library that has problems being deployed in a zipfile will have problems in other places besides easy_install. The fix isn't to stop using zipfiles, it's to fix the places in Python (and 3rd-party tools) where zipfile support isn't finished. easy_install simply forces the errors to show up earlier... and for the community as a whole, that's a *good* thing. (Oh, and while we're suggesting things, zooko, I'd suggest you take a look at the two setuptools bugs in "testing" status that are assigned to you; they're holding up the 0.6c9 release.) At 11:27 AM 9/10/2008 -0600, zooko wrote:
On Sep 10, 2008, at 5:13 AM, skip@pobox.com wrote:
If I have a package installed as a zip file egg and ask for its __file__ I might get a reference which looks like a normal file system path but isn't:
I advise people to add to their distutils config file:
[easy_install] zip_ok=False
In addition, I suggest that setuptools should be changed to install eggs unzipped by default, as there is a list of problems and inconveniences with zipping eggs and, as far as I am aware, no measured benefits.
Here is the issue ticket, in which your problem, Skip, is the most recent entry in the list:
http://bugs.python.org/setuptools/issue33
Regards,
Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
On Sep 11, 2008, at 10:03 AM, Phillip J. Eby wrote:
(Oh, and while we're suggesting things, zooko, I'd suggest you take a look at the two setuptools bugs in "testing" status that are assigned to you; they're holding up the 0.6c9 release.)
I will be able to look at those next week. Regards, Zooko --- http://allmydata.org -- Tahoe, the Least-Authority Filesystem http://allmydata.com -- back up all your files for $5/month
On Thu, 2008-09-11 at 12:03 -0400, Phillip J. Eby wrote:
A library that has problems being deployed in a zipfile will have problems in other places besides easy_install.
My problems with libraries in zipfiles have generally revolved around limited user accounts that cannot handle the unzip. A normal python installation provides files than can be used directly. So I've configured the distutils configuration file to always unzip. I've found it easier for the systems I administer to force all installations to be unzipped rather than trying to find those accounts that will have problems with zipped libraries. These errors can be pretty mysterious to a sysadmin who is not fluent in Python. It can also be pretty embarrassing when the test system works and the live system fails because of the change in user account. If there is a better/smarter approach for installing zipped libraries I'd be happy to use it. -- Lloyd Kvam Venix Corp DLSLUG/GNHLUG library http://dlslug.org/library.html http://www.librarything.com/catalog/dlslug http://www.librarything.com/rsshtml/recent/dlslug http://www.librarything.com/rss/recent/dlslug
participants (5)
-
Fadhley Salim -
Phillip J. Eby -
Python -
skip＠pobox.com -
zooko