Hi, There's buildout 1.4.4, 1.5.2 and an alpha of 2.0. The 2.0 seems to aim at python 3 and is, according to the docs, in a very early alpha stage, so I'll leave that one out. What I'm using now is 1.4.4 with the special bootstrap.py that ensures it picks 1.4.4 instead of a newer one. The reason: it works with http://pypi.python.org/pypi/osc.recipe.sysegg which finds system wide python packages (numpy and other stuff better installed through the OS). I tried 1.5.x half a year ago, but it didn't find numpy&friends on ubuntu even though 1.5 is supposed to allow just that. I tried it this morning on OSX with 1.5.2 but still no luck. It doesn't want to find the OS-level packages. But... sticking to ye olde 1.4.4 seems wrong to me. So I phoned my brother Maurits (hi!) who is working with plone: plone is a heavy buildout user, so what's the status there? He said plone is basically also still on 1.4.4! a) I feel a bit better sticking to 1.4.4, knowing that plone also does it. b) Is anyone actually using 1.5.2 with system packages? And by that I mean that you have a dependency on something like numpy and buildout actually finds that package locally and won't try and grab it off pypi? Am I doing something wrong? I dare say I've got an above average experience with buidout and I just can't get this to work. Reinout p.s.: Here's my buildout config. Am I doing something completely wrong? [buildout] parts = scripts include-site-packages = true allowed-eggs-from-site-packages = numpy [scripts] recipe = zc.recipe.egg eggs = numpy -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On Tue, Aug 16, 2011 at 7:45 AM, Reinout van Rees <reinout@vanrees.org> wrote:
But... sticking to ye olde 1.4.4 seems wrong to me. So I phoned my brother Maurits (hi!) who is working with plone: plone is a heavy buildout user, so what's the status there? He said plone is basically also still on 1.4.4!
We're mostly using zc.buildout 1.4.4 at Zope Corporation (home of zc.buildout) as well. Except where we're using older version (which will get updated before too much longer). Buildout 1.5.x introduced features we just haven't needed. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> "A person who won't read has no advantage over one who can't read." --Samuel Langhorne Clemens
Hello, On Tue, Aug 16, 2011 at 13:45, Reinout van Rees <reinout@vanrees.org> wrote:
b) Is anyone actually using 1.5.2 with system packages? And by that I mean that you have a dependency on something like numpy and buildout actually finds that package locally and won't try and grab it off pypi?
Yes, we use zc.buildout-1.5.2 with a project that runs Pyramid on Google's App Engine. I run the development environment in Gentoo, and I have net-zope/zope-interface installed in /usr/lib64/python?.?/site-packages/zope/interface. When I run buildout with a system Python that's ABI is supported by the zope-inteface ebuild, it will use the one from the system site-packages. IOW, worksforme. Have you tried the allowed-eggs-from-site-packages [1] option? [1] http://pypi.python.org/pypi/zc.buildout/1.5.2#working-with-a-system-python Attila
On 16-08-11 14:41, Attila Oláh wrote:
Hello,
On Tue, Aug 16, 2011 at 13:45, Reinout van Rees<reinout@vanrees.org> wrote:
b) Is anyone actually using 1.5.2 with system packages? And by that I mean that you have a dependency on something like numpy and buildout actually finds that package locally and won't try and grab it off pypi?
Yes, we use zc.buildout-1.5.2 with a project that runs Pyramid on Google's App Engine. I run the development environment in Gentoo, and I have net-zope/zope-interface installed in /usr/lib64/python?.?/site-packages/zope/interface. When I run buildout with a system Python that's ABI is supported by the zope-inteface ebuild, it will use the one from the system site-packages. IOW, worksforme.
Just making sure: you have a dependency on zope.interface in your setup.py's install_requires? So that buildout actively searches for it? (ABI compatibility... I *did* upgrade to OSX Lion and installing some things give me an "cannot find gcc-4.0" error. But when running my sites, mapnik and the other libraries work just fine, so I think that shouldn't be the problem. But I don't know what buildout 1.5 does in detail.)
Have you tried the allowed-eggs-from-site-packages [1] option?
Yes. Didn't have any effect, it seemed. Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On 16/08/2011 04:45, Reinout van Rees wrote:
Hi,
There's buildout 1.4.4, 1.5.2 and an alpha of 2.0.
I've used 1.5.2 happily for ages, but I don't use the stuff Gary introduced there... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Tue, Aug 16, 2011 at 8:10 AM, Fred Drake <fdrake@acm.org> wrote:
On Tue, Aug 16, 2011 at 7:45 AM, Reinout van Rees <reinout@vanrees.org> wrote:
But... sticking to ye olde 1.4.4 seems wrong to me. So I phoned my brother Maurits (hi!) who is working with plone: plone is a heavy buildout user, so what's the status there? He said plone is basically also still on 1.4.4!
We're mostly using zc.buildout 1.4.4 at Zope Corporation (home of zc.buildout) as well. Except where we're using older version (which will get updated before too much longer).
Buildout 1.5.x introduced features we just haven't needed.
I use 1.5. :) I don't personally care about integration with system Python or system packages. I do plan to keep maintaining 1.5. I don't plan maintenance on 1.4. Of course, I plan to support 2x over the long term. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton
On Tue, Aug 16, 2011 at 15:40, Reinout van Rees <reinout@vanrees.org> wrote:
On 16-08-11 14:41, Attila Oláh wrote:
Hello,
On Tue, Aug 16, 2011 at 13:45, Reinout van Rees<reinout@vanrees.org> wrote:
b) Is anyone actually using 1.5.2 with system packages? And by that I mean that you have a dependency on something like numpy and buildout actually finds that package locally and won't try and grab it off pypi?
Yes, we use zc.buildout-1.5.2 with a project that runs Pyramid on Google's App Engine. I run the development environment in Gentoo, and I have net-zope/zope-interface installed in /usr/lib64/python?.?/site-packages/zope/interface. When I run buildout with a system Python that's ABI is supported by the zope-inteface ebuild, it will use the one from the system site-packages. IOW, worksforme.
Just making sure: you have a dependency on zope.interface in your setup.py's install_requires? So that buildout actively searches for it?
Good point. It wasn't there, I only listed "pyramid" and a bunch of other packages. Let's add it and experiment. Let's update zope-interface first: $ sudo emerge --update --oneshot zope-interface [...] * * Python namespaces: * '/usr/lib64/python2.6/site-packages/zope.interface-3.7.0-py2.6-nspkg.pth': * zope * '/usr/lib64/python3.1/site-packages/zope.interface-3.7.0-py3.1-nspkg.pth': * zope * '/usr/lib64/python2.4/site-packages/zope.interface-3.7.0-py2.4-nspkg.pth': * zope * '/usr/lib64/python2.7/site-packages/zope.interface-3.7.0-py2.7-nspkg.pth': * zope * '/usr/lib64/python3.2/site-packages/zope.interface-3.7.0-py3.2-nspkg.pth': * zope * '/usr/lib64/python2.5/site-packages/zope.interface-3.7.0-py2.5-nspkg.pth': * zope * [...] Now I update setup.py, adding a few packages just for testing: $ git clone vemble tmp && cd tmp $ vim setip.py $ grep req setup.py install_requires=('zope.interface', 'numpy', 'scipy', 'nose', 'six'), Note that I'll use a simplified buildout.cfg for testing: $ cat buildout.cfg [buildout] develop = . parts = graph unzip = true [graph] recipe = z3c.recipe.depgraph eggs = vemble variants = tred strict = true Check Python version: $ python -V Python 2.7.2+ Note that I have zope.interface, numpy, scipy and nose on my systme. I do not have 'six' though. $ python -c "print __import__('zope.interface').interface" <module 'zope.interface' from '/usr/lib64/python2.7/site-packages/zope/interface/__init__.pyc'> $ python -c "print __import__('numpy')" <module 'numpy' from '/usr/lib64/python2.7/site-packages/numpy/__init__.pyc'> $ python -c "print __import__('scipy')" <module 'scipy' from '/usr/lib64/python2.7/site-packages/scipy/__init__.pyc'> $ python -c "print __import__('nose')" <module 'nose' from '/usr/lib64/python2.7/site-packages/nose/__init__.pyc'> $ python bootstrap.py --distribute [...] $ ls eggs zc.buildout-1.5.2-py2.7.egg $ ./bin/buildout [...] Egg from site-packages: distribute 0.6.19 [...] Getting distribution for 'six'. Running easy_install: [...] Got six 1.0.0. Picked: six = 1.0.0 [...] Getting required 'nose' required by vemble 0. We have the best distribution that satisfies 'nose'. Egg from site-packages: nose 1.1.2 Getting required 'scipy' required by vemble 0. We have the best distribution that satisfies 'scipy'. Egg from site-packages: scipy 0.9.0 Getting required 'numpy' required by vemble 0. We have the best distribution that satisfies 'numpy'. Egg from site-packages: numpy 1.6.1 Getting required 'zope.interface' required by vemble 0. We have the best distribution that satisfies 'zope.interface'. Egg from site-packages: zope.interface 3.7.0 $ ls -1 eggs six-1.0.0-py2.7.egg tl.eggdeps-0.4-py2.7.egg z3c.recipe.depgraph-0.5-py2.7.egg zc.buildout-1.5.2-py2.7.egg zc.recipe.egg-1.3.2-py2.7.egg $ cat bin/graph-graph [...] import sys sys.path[0:0] = [ '/home/aatiis/repos/tmp/src', '/home/aatiis/repos/tmp/eggs/z3c.recipe.depgraph-0.5-py2.7.egg', '/home/aatiis/repos/tmp/eggs/tl.eggdeps-0.4-py2.7.egg', '/home/aatiis/repos/tmp/eggs/zc.recipe.egg-1.3.2-py2.7.egg', '/home/aatiis/repos/tmp/eggs/zc.buildout-1.5.2-py2.7.egg', '/usr/lib64/python2.7/site-packages', '/usr/lib64/python2.7/site-packages', '/usr/lib64/python2.7/site-packages', '/usr/lib64/python2.7/site-packages', '/usr/lib64/python2.7/site-packages', '/home/aatiis/repos/tmp/eggs/six-1.0.0-py2.7.egg', ] [...] As you can see, it uses '/usr/lib64/python2.7/site-packages' multiple times, for all the packages that I have on my system. Note that I used to have some problems with this on Ubuntu, but with Gentoo it works just fine.
(ABI compatibility... I *did* upgrade to OSX Lion and installing some things give me an "cannot find gcc-4.0" error. But when running my sites, mapnik and the other libraries work just fine, so I think that shouldn't be the problem. But I don't know what buildout 1.5 does in detail.)
I have the following ABIs enabled: $ eix zope-interface | grep -i installed Installed versions: 3.7.0{tbz2}[1](16:37:53 16/08/11)(python_abis_2.4 python_abis_2.5 python_abis_2.6 python_abis_2.7 python_abis_3.1 python_abis_3.2 -python_abis_2.5-jython -python_abis_2.7-pypy-1.5 -python_abis_3.3) This basically means that zope.interface is installed in the site-packages of my Python 2.4 (though it fails with this version), 2.5, 2.6, 2.7, 3.1 and 3.2, and not installed for Python 3.3, Jython and PyPy. Also note that Gentoo correctly installs these Python packages as eggs, including their .egg-info directory. I hope I could be of some help! Cheers, Attila
Have you tried the allowed-eggs-from-site-packages [1] option?
Yes. Didn't have any effect, it seemed.
Reinout
-- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
On 16-08-11 17:04, Attila Oláh wrote:
Getting required 'nose' required by vemble 0. We have the best distribution that satisfies 'nose'. Egg from site-packages: nose 1.1.2 Getting required 'scipy' required by vemble 0. We have the best distribution that satisfies 'scipy'. Egg from site-packages: scipy 0.9.0 Getting required 'numpy' required by vemble 0. We have the best distribution that satisfies 'numpy'. Egg from site-packages: numpy 1.6.1 Getting required 'zope.interface' required by vemble 0. We have the best distribution that satisfies 'zope.interface'. Egg from site-packages: zope.interface 3.7.0
That's pretty convincing, thanks for trying it out! So on gentoo it works. On Ubuntu it doesn't (at least half a year ago, but that was already with 1.5.2 iirc) and on OSX it doesn't (as far as I can see). Anyone on OSX or ubuntu who got it to work? I'd dearly like to get it to work, too. Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On Tue, Aug 16, 2011 at 13:45, Reinout van Rees <reinout@vanrees.org> wrote:
recipe = zc.recipe.egg
zc.recipe.egg can handle the new buildout options? When I want system packages I use z3.recipe.scripts : http://pypi.python.org/pypi/z3c.recipe.scripts/ -- Sebastien Douche <sdouche@gmail.com> Twitter : @sdouche
On 16-08-11 17:46, Sebastien Douche wrote:
recipe = zc.recipe.egg zc.recipe.egg can handle the new buildout options? When I want system
On Tue, Aug 16, 2011 at 13:45, Reinout van Rees<reinout@vanrees.org> wrote: packages I use z3.recipe.scripts : http://pypi.python.org/pypi/z3c.recipe.scripts/
I'm pretty sure zc.recipe.egg can, as it is bundled with buildout itself! But I *did* test out z3c.recipe.scripts and I had the same problem: buildout would try and build numpy from source. And numpy for sure is installed (and osc.recipe.sysegg finds it just fine): $ python Python 2.6.6 (r266:84374, Aug 31 2010, 11:00:51) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "help", "copyright", "credits" or "license" for more information.
import numpy numpy.__file__
'/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/__init__.pyc' And my bin/buildout is: $ cat bin/buildout #!/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python -S import sys sys.path[0:0] = [ '/private/tmp/numpytest/parts/buildout', ] import os path = sys.path[0] if os.environ.get('PYTHONPATH'): path = os.pathsep.join([path, os.environ['PYTHONPATH']]) os.environ['BUILDOUT_ORIGINAL_PYTHONPATH'] = os.environ.get('PYTHONPATH', '') os.environ['PYTHONPATH'] = path import site # imports custom buildout-generated site.py import zc.buildout.buildout if __name__ == '__main__': zc.buildout.buildout.main() And just to make sure, here's my buildout.cfg: [buildout] parts = scripts include-site-packages = true allowed-eggs-from-site-packages = numpy [scripts] recipe = zc.recipe.egg eggs = numpy Anyone spot anything suspicious? Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On 16/08/2011 08:56, Reinout van Rees wrote:
Anyone spot anything suspicious?
Maybe pin numpy to the version provided by the system? Just a guess and nothing I've tried... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Hi, I'm didn't follow the full thread so I apologize if I'm off topic If I remember correctly I had a couple of issues last year with numpy/ matplotlib and buildout because the two first weren't compliant with python packages. I think numpy got python packaging compatibility for 1.5.0 so if your numpy's system wide installation is lower than that you most probably have no python package information hence Regards, Xavier.
On 17-08-11 08:16, Xavier Ordoquy wrote:
If I remember correctly I had a couple of issues last year with numpy/ matplotlib and buildout because the two first weren't compliant with python packages. I think numpy got python packaging compatibility for 1.5.0 so if your numpy's system wide installation is lower than that you most probably have no python package information hence
I've got 1.5.1 with a proper numpy-1.5.1-py2.6.egg-info directory in the site-packages. But I also didn't trust numpy completely :-) , so I also did the tests with psycopg2: same result. Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On 17-08-11 04:23, Chris Withers wrote:
Maybe pin numpy to the version provided by the system?
Just a guess and nothing I've tried...
*Partial hurray sound* Actually... that works. If I pin numpy to 1.5.1, buildout uses my site-packages one! We're one step further! Next hurdle: I don't want to pin that package, as I don't particularly care which numpy version it is as long as it is recent enough. If I pin 1.5.12, it'll be fine on my OSX machine, but ubuntu probably has a slightly different version. And our webserver might be hanging one ubuntu version back. So: is there a way to *force* buildout to use the site-packages one, regardless of the version? (Provided it matches some minimum version requirement in case I provided one, of course). I can't find it in the docs. osc.recipe.sysegg has a "force-sysegg" option that does just that. Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
participants (7)
-
Attila Oláh -
Chris Withers -
Fred Drake -
Jim Fulton -
Reinout van Rees -
Sebastien Douche -
Xavier Ordoquy