Mailman 3 January 2006 - Distutils-SIG

SOURCES not working for me
by Ian Bicking 25 Jan '06

25 Jan '06

I'm stumped as to why this package is not working properly: http://svn.pythonpaste.org/RhubarbTart/trunk The package data isn't being included when I build an egg (everything in rhubarbtart/paste_templates/). But SOURCES.txt lists all the proper files. I'm quite baffled. -- Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org

2 1

Re: [Distutils] Problem upgrading docutils with setuptools
by Phillip J. Eby 25 Jan '06

25 Jan '06

At 06:46 PM 1/25/2006 +0100, Maciek Starzyk wrote: >Is there a way to tell setup.py to do the equivalent of easy_install -m >while upgrading its dependencies ? No, not really. You could always just bundle the "roman" module in your project, I suppose, or create a package for it on PyPI and make it a requirement of your project. These approaches wouldn't change docutils' behavior, but it'd ensure that *your* projects using docutils would work.

3 4

Re: [Distutils] Problem upgrading docutils with setuptools
by Phillip J. Eby 25 Jan '06

25 Jan '06

At 02:53 PM 1/25/2006 +0100, Maciek Starzyk wrote: >Then upgrade to docutils 0.4. >Note that setup says "roman" module is already there. It was installed >as a part of docutils 0.3.9. At the moment when setup checks for >module roman - docutils 0.3.9 egg is still on the PYTHONPATH and therefore >"roman" is not installed. A simple fix is to do "easy_install -m docutils" before the upgrade, as this will remove the existing version from the .pth file. > But later on docutils 0.3.9 egg is removed from >PYTHONPATH and docutils 0.4 fails: > >EasyInstall docs say: > > installing a package automatically replaces > > any previous version in the easy-install.pth file > >So maybe this replacing should be done earlier - ensuring that the >previous version's code >is not in path - before actually running setup ? > >Or is it a bug in docutils' setup.py ? Well, docutils' setup.py is clearly trying to support Python versions older than 2.3, or else it would not be checking for textwrap and optparse. Setuptools only supports 2.3 and higher. So those modules shouldn't be an issue. If docutils were intended to use setuptools, then it would not make sense to check for 'roman', either, since an external dependency should be just that, a dependency. So in that case I would consider it a bug in their setup.py. However, since they clearly are trying to support 2.2 and maybe older versions as well, it's hard to call it that. Rather, I'd have to say it's a limitation of setuptools' ability to be backward-compatible with highly-customized setup scripts. Unfortunately, there is no way to remove the existing version from sys.path before the setup script runs, as before setup() is called, there's no way to know precisely which project needs to be removed. EasyInstall accepts URLs and directory names as well as project names, so it's not guaranteed that it will know what project it's building. EasyInstall also doesn't change sys.path at all under normal circumstances; it only changes the .pth file so that sys.path will be changed the next time Python is started. If I were to propose a change to docutils' setup.py, I would suggest that roman.py should either be bundled as part of the docutils package (i.e. always installed, but inside the docutils package directory), or else be treated as an external dependency (never installed by the setup script directly, only by the user or by setuptools' dependency handling). Either of these approaches is a robust way to deal with the issue. Checking for modules' presence at installation time, however, isn't a good practice at all; I found it to be fragile and error-prone for PEAK (which used to rely on zope.interface, among other things) before setuptools even existed.

3 4

setuptools include_package_data ignore data under a 'package'?
by bbangert＠groovie.org 25 Jan '06

25 Jan '06

I've been having some strange issues when trying to get include_package_data to actually include package data, when said data is under a module. Though it seems to behave unpredictably. For example, several people have installed Pylons and its been missing pylons/media/style/orange.css (a package file), though I haven't had this happen myself. However, when I make a new template project, then bundle it into a distribution and unpack it (to see what setuptools thought was package data). It's missing several directories. Easy way to reproduce: paster create --template=pylons test cd test python setup.py sdist cd dist tar xzvf test-0.0.0dev.tar.gz Then compare: ls test-0.0.0dev/test/ To ../test/ The new one is missing the docs, public, templates directories which are package data directories containing content. If these aren't included on purpose, where can I get more information on exactly what include_package_data=True does actually include? BTW, I'm using setuptools 0.6a9 Thanks, Ben

4 6

Problem upgrading docutils with setuptools
by Maciek Starzyk 25 Jan '06

25 Jan '06

Hi. Recently I had some troubles with clean update of docutils. 0.3.9 -> 0.4 It looks to me like a setuptools problem. See below: First install docutils 0.3.9: ----------------------- 8< ---------------------------------- D:\download\python\docutils>easy_install docutils-0.3.9.tar.gz Processing docutils-0.3.9.tar.gz Running docutils-0.3.9\setup.py -q bdist_egg --dist-dir c:\docume~1\mast\ustawi~1\temp\easy_install-cxyfp2\docutils-0.3.9\egg-dist-tmp-rcp8-e "optparse" module already present; ignoring extras/optparse.py. "textwrap" module already present; ignoring extras/textwrap.py. zip_safe flag not set; analyzing archive contents... Adding docutils 0.3.9 to easy-install.pth file Installing rst2latex.py script to C:\Python24\Scripts Installing rst2html.py script to C:\Python24\Scripts Installed c:\python24\lib\site-packages\docutils-0.3.9-py2.4.egg Processing dependencies for docutils==0.3.9 D:\download\python\docutils>python -c "import docutils.parsers.rst; print 'OK'" OK ----------------------- 8< ---------------------------------- Then upgrade to docutils 0.4. Note that setup says "roman" module is already there. It was installed as a part of docutils 0.3.9. At the moment when setup checks for module roman - docutils 0.3.9 egg is still on the PYTHONPATH and therefore "roman" is not installed. But later on docutils 0.3.9 egg is removed from PYTHONPATH and docutils 0.4 fails: ----------------------- 8< ---------------------------------- D:\download\python\docutils>easy_install -U docutils-0.4.tar.gz Processing docutils-0.4.tar.gz Running docutils-0.4\setup.py -q bdist_egg --dist-dir c:\docume~1\mast\ustawi~1\ temp\easy_install-hnciga\docutils-0.4\egg-dist-tmp-y4g2fn "optparse" module already present; ignoring extras/optparse.py. "textwrap" module already present; ignoring extras/textwrap.py. "roman" module already present; ignoring extras/roman.py. zip_safe flag not set; analyzing archive contents... docutils.parsers.rst.directives.misc: module references __file__ docutils.writers.html4css1.__init__: module references __file__ docutils.writers.newlatex2e.__init__: module references __file__ docutils.writers.pep_html.__init__: module references __file__ docutils.writers.s5_html.__init__: module references __file__ Removing docutils 0.3.9 from easy-install.pth file Adding docutils 0.4 to easy-install.pth file Installing rst2html.py script to C:\Python24\Scripts Installing rst2latex.py script to C:\Python24\Scripts Installing rst2newlatex.py script to C:\Python24\Scripts Installing rst2pseudoxml.py script to C:\Python24\Scripts Installing rst2s5.py script to C:\Python24\Scripts Installing rst2xml.py script to C:\Python24\Scripts Installed c:\python24\lib\site-packages\docutils-0.4-py2.4.egg Processing dependencies for docutils==0.4 D:\download\python\docutils>python -c "import docutils.parsers.rst; print 'OK'" Traceback (most recent call last): File "<string>", line 1, in ? File "c:\python24\lib\site-packages\docutils-0.4-py2.4.egg\docutils\parsers\rst\__init__.py", line 77, in ? from docutils.parsers.rst import states File "c:\python24\lib\site-packages\docutils-0.4-py2.4.egg\docutils\parsers\rst\states.py", line 110, in ? import roman ImportError: No module named roman ----------------------- 8< ---------------------------------- EasyInstall docs say: > installing a package automatically replaces > any previous version in the easy-install.pth file So maybe this replacing should be done earlier - ensuring that the previous version's code is not in path - before actually running setup ? Or is it a bug in docutils' setup.py ? Cheers, Maciek

1 0

Re: [Distutils] no access to resources unpacked as root
by Phillip J. Eby 23 Jan '06

23 Jan '06

At 12:42 AM 1/24/2006 +0000, Matt Goodall wrote: >The Twisted process starts up as root to open low ports, etc and then, >once up and running, sheds privileges and continues to run as a normal >user. Many of the resources that are buried in the eggs are located with >resource_filename at module import time; some when the application is >still running as root. > >The resources are unpacked from the egg into /root/.python-eggs/ and >(correctly, I think) given 600 (rw-------) file access permissions. The >problem is that the user the application ends up running as cannot access >the unpacked resource files and fails with a PermissionDenied error. > >Now, I can move some uses of resource_filename to happens later but I >don't think I can replace all of them without changing APIs. > >Has anyone else tackled this problem or can anyone offer any advice on how >to avoid the problem in the first place? The simplest approach is probably to mark the eggs as not zip-safe, or to configure easy_install to install them unzipped.

1 0

no access to resources unpacked as root
by Matt Goodall 23 Jan '06

23 Jan '06

Hi, I'm using eggs a fair amount to deploy Twisted-based server applications. (This problem is not specific to Twisted so please don't stop reading!) The Twisted process starts up as root to open low ports, etc and then, once up and running, sheds privileges and continues to run as a normal user. Many of the resources that are buried in the eggs are located with resource_filename at module import time; some when the application is still running as root. The resources are unpacked from the egg into /root/.python-eggs/ and (correctly, I think) given 600 (rw-------) file access permissions. The problem is that the user the application ends up running as cannot access the unpacked resource files and fails with a PermissionDenied error. Now, I can move some uses of resource_filename to happens later but I don't think I can replace all of them without changing APIs. Has anyone else tackled this problem or can anyone offer any advice on how to avoid the problem in the first place? Thanks in advance, - Matt

1 0

using setup.py to uninstall
by John M. Gabriele 20 Jan '06

20 Jan '06

Can distutils (setup.py) uninstall packages as well? If not, is this a feature that's being worked on, or is there some reason why it may not be necessary? Thanks, ---J __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

3 2

Re: [Distutils] namespace_packages
by Phillip J. Eby 20 Jan '06

20 Jan '06

At 12:44 PM 01/20/2006 -0500, Charlie Moad wrote: >Here are the eggs if you want to unzip and look at them. I installed >both unzipped. > >http://euclid.uits.iupui.edu/~cmoad/mpleggs/matplotlib-0.86.2cvs-py2.4-linu… >http://euclid.uits.iupui.edu/~cmoad/mpleggs/basemap-0.8-py2.4-linux-i686.egg There are two issues that I saw. First, the matplotlib egg above does not declare a namespace in its matplotlib/__init__.py. Second, the basemap egg does not contain a matplotlib/__init__.py at all. Please note that you *must* include an __init__.py file for every namespace package and parent package thereof, in *every* egg that's part of the namespace package, and that these __init__.py files must all contain a namespace declaration. Missing even one can cause problems. For example, the basemap egg is simply not importable, because as far as Python is concerned it does not contain a matplotlib package (due to the missing __init__.py).

2 1

Re: [Distutils] namespace_packages
by Phillip J. Eby 20 Jan '06

20 Jan '06

At 11:20 AM 01/20/2006 -0500, Charlie Moad wrote: >On 1/19/06, Robert Kern <robert.kern(a)gmail.com> wrote: > > Charlie Moad wrote: > > > Well here are th cvs links to them. > > > > > > > http://cvs.sourceforge.net/viewcvs.py/matplotlib/matplotlib/setup.py?rev=1.… > > > > http://cvs.sourceforge.net/viewcvs.py/matplotlib/toolkits/basemap/setup.py?… > > > > > > The matplotlib.toolkits module does not exist in the mpl cvs and it is > > > not in the setup.py file. I have tried adding both. In basemap > > > however, the matplotlib.toolkits module does exist and is listed in > > > the setup.py file. > > > > Both of these setup.py files are buggy. The packages list needs to have > entries > > in dotted form ("matplotlib.toolkits"), not filesystem form > > ("matplotlib/toolkits"). Correcting them, and adding an empty > > lib/matplotlib/toolkits/__init__.py to the matplotlib checkout allows me to > > build eggs with namespace_packages set appropriately. > > > >I finally got time to try this, and it didn't help much. Changing all >the packages to dot notation works fine. I have added the >toolkits/__init__.py file. Setuptools warns about not having >declare_namespace in the init files. I have added this: > >try: > __import__('pkg_resources').declare_namespace(__name__) > print __name__, __file__ >except ImportError: > pass # must not have setuptools Note that if you don't have pkg_resources, namespace packages simply won't work, so there's little point in the try/except. >to __init__.py in mpl's and basemap's matplotlib and >matplotlib.toolkits module, hence in 4 places. >Upon starting python I get this output: > >matplotlib >/usr/lib/python2.4/site-packages/matplotlib-0.86.2cvs-py2.4-linux-i686.egg/matplotlib/__init__.pyc >matplotlib.toolkits >/usr/lib/python2.4/site-packages/matplotlib-0.86.2cvs-py2.4-linux-i686.egg/matplotlib/toolkits/__init__.pyc > >So the basemap declarations are never being reached. Where should I >go from here? Is the other egg you're looking for on sys.path? >_______________________________________________ >Distutils-SIG maillist - Distutils-SIG(a)python.org >http://mail.python.org/mailman/listinfo/distutils-sig

2 2

Distutils-SIG January 2006