Buildout 2.0.0 released
I'm very happy, finally, to have released buildout 2.0.0. Much has changed from buildout 1: http://pypi.python.org/pypi/zc.buildout/2.0.0#id3 Installation instructions: http://pypi.python.org/pypi/zc.buildout/2.0.0#installation Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On Sun, 2013-02-10 at 13:24 -0500, Jim Fulton wrote:
I'm very happy, finally, to have released buildout 2.0.0.
Much has changed from buildout 1:
http://pypi.python.org/pypi/zc.buildout/2.0.0#id3
Installation instructions:
Woot! - C
Nice ! Congratulation to everyone !
On Sun, Feb 10, 2013 at 2:19 PM, Chris McDonough
On Sun, 2013-02-10 at 13:24 -0500, Jim Fulton wrote:
I'm very happy, finally, to have released buildout 2.0.0.
Much has changed from buildout 1:
http://pypi.python.org/pypi/zc.buildout/2.0.0#id3
Installation instructions:
Woot!
- C
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Mathieu Leduc-Hamel Senior Developer at Ajah
Hello, That caused a severe heartattack on winbot. :-S Any advices what to do? On 02/10/2013 07:24 PM, Jim Fulton wrote:
I'm very happy, finally, to have released buildout 2.0.0.
Much has changed from buildout 1:
http://pypi.python.org/pypi/zc.buildout/2.0.0#id3
Installation instructions:
http://pypi.python.org/pypi/zc.buildout/2.0.0#installation
Jim
-- Best regards, Adam GROSZER -- Quote of the day: Your supervisor is thinking about you.
On 11-02-13 08:46, Adam GROSZER wrote:
That caused a severe heartattack on winbot. :-S Any advices what to do?
I *think* the basic solution is to use a newer bootstrap. buildout 1.7 and 2.0 include a build-in version restriction to <2.0 and
=2.0 respectively to prevent accidental updates. The latest bootstraps use that.
http://downloads.buildout.org/1/bootstrap.py http://downloads.buildout.org/2/bootstrap.py Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On 11/02/2013 09:02, Reinout van Rees wrote:
On 11-02-13 08:46, Adam GROSZER wrote:
That caused a severe heartattack on winbot. :-S Any advices what to do?
I *think* the basic solution is to use a newer bootstrap.
buildout 1.7 and 2.0 include a build-in version restriction to <2.0 and
=2.0 respectively to prevent accidental updates. The latest bootstraps use that.
http://downloads.buildout.org/1/bootstrap.py http://downloads.buildout.org/2/bootstrap.py
But we can drop the -t for buildout 2 now, right? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On 11-02-13 10:29, Chris Withers wrote:
I *think* the basic solution is to use a newer bootstrap.
buildout 1.7 and 2.0 include a build-in version restriction to <2.0 and
=2.0 respectively to prevent accidental updates. The latest bootstraps use that.
http://downloads.buildout.org/1/bootstrap.py http://downloads.buildout.org/2/bootstrap.py
But we can drop the -t for buildout 2 now, right?
*checking* Yes, we can! Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On 02/11/2013 10:02 AM, Reinout van Rees wrote:
On 11-02-13 08:46, Adam GROSZER wrote:
That caused a severe heartattack on winbot. :-S Any advices what to do?
I *think* the basic solution is to use a newer bootstrap.
buildout 1.7 and 2.0 include a build-in version restriction to <2.0 and
=2.0 respectively to prevent accidental updates. The latest bootstraps use that.
http://downloads.buildout.org/1/bootstrap.py http://downloads.buildout.org/2/bootstrap.py
Reinout
Yay! That did it! -- Best regards, Adam GROSZER -- Quote of the day: It is not good to be too free. It is not good to have everything one wants. - Blaise Pascal
On Mon, Feb 11, 2013 at 2:46 AM, Adam GROSZER
Hello,
That caused a severe heartattack on winbot. :-S
Could you be more specific? Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On 10-02-13 19:24, Jim Fulton wrote:
I'm very happy, finally, to have released buildout 2.0.0.
What's being picked: zc.buildout = 2.0.0 zc.recipe.egg = 2.0.0a3 Shouldn't zc.recipe.egg be tagged as 2.0.0 final, too? Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On Mon, Feb 11, 2013 at 4:46 AM, Reinout van Rees
On 10-02-13 19:24, Jim Fulton wrote:
I'm very happy, finally, to have released buildout 2.0.0.
What's being picked:
zc.buildout = 2.0.0 zc.recipe.egg = 2.0.0a3
Shouldn't zc.recipe.egg be tagged as 2.0.0 final, too?
Yes. I'll do that. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
Jim Fulton
I'm very happy, finally, to have released buildout 2.0.0.
I'm happy too, congrats to everybody contributed!
Much has changed from buildout 1:
* Buildout no-longer tries to provide full or partial isolation from
system Python installations. If you want isolation, use buildout with
virtualenv, or use a clean build of Python to begin with.
Ok, I think I'm hitting this, trying to update one project: one of its
scripts fails with the following traceback::
$ .../bin/initialize_cpi_db --development src/cpi/development.ini
Traceback (most recent call last):
File ".../bin/initialize_cpi_db", line 51, in <module>
import cpi.scripts.initializedb
File ".../src/cpi/cpi/__init__.py", line 10, in <module>
from pyramid.config import Configurator
File ".../eggs/pyramid-1.4-py2.7.egg/pyramid/config/__init__.py", line 21, in <module>
from pyramid.authorization import ACLAuthorizationPolicy
File ".../eggs/pyramid-1.4-py2.7.egg/pyramid/authorization.py", line 9, in <module>
from pyramid.security import (
File ".../eggs/pyramid-1.4-py2.7.egg/pyramid/security.py", line 12, in <module>
from pyramid.threadlocal import get_current_registry
File ".../eggs/pyramid-1.4-py2.7.egg/pyramid/threadlocal.py", line 3, in <module>
from pyramid.registry import global_registry
File ".../eggs/pyramid-1.4-py2.7.egg/pyramid/registry.py", line 5, in <module>
from zope.interface.registry import Components
ImportError: No module named registry
And the problem is that, even if I explicitly said it needs the
zope.interface egg, and pinned it to version 4.0.3, the script sees the
system wide zope.interface installed by the underlying Ubuntu host::
$ cat .../bin/initialize_cpi_db
#!/usr/bin/python
import sys
sys.path[0:0] = [
'.../eggs/SQLAlchemy-0.8.0b2-py2.7-linux-i686.egg',
'.../src/cpi',
'.../eggs/docutils-0.10-py2.7.egg',
'.../eggs/logutils-0.3.3-py2.7.egg',
'.../src/metapensiero.extjs.desktop/src',
'.../src/metapensiero.sphinx.patchdb/src',
'.../src/metapensiero.sqlalchemy.proxy/src',
'.../eggs/psycopg2-2.4.6-py2.7-linux-i686.egg',
'.../eggs/pyramid-1.4-py2.7.egg',
'.../eggs/pyramid_beaker-0.7-py2.7.egg',
'.../eggs/pyramid_mailer-0.10-py2.7.egg',
'.../eggs/rst2pdf-0.93.dev-py2.7.egg',
'.../eggs/simplejson-3.0.7-py2.7-linux-i686.egg',
'.../eggs/Babel-1.0dev_r661-py2.7.egg',
'.../eggs/Sphinx-1.1.3-py2.7.egg',
'.../eggs/coverage-3.5.1-py2.7-linux-i686.egg',
'.../eggs/lingua-1.4-py2.7.egg',
'.../eggs/nose-1.2.1-py2.7.egg',
'.../eggs/nose_progressive-1.4-py2.7.egg',
'.../eggs/yuicompressor-2.4.7-py2.7.egg',
'.../eggs/blessings-1.3-py2.7.egg',
'.../eggs/xlwt-0.7.4-py2.7.egg',
'.../eggs/xlrd-0.8.0-py2.7.egg',
'.../eggs/polib-1.0.2-py2.7.egg',
'.../eggs/Jinja2-2.6-py2.7.egg',
'.../eggs/Pygments-1.5-py2.7.egg',
'.../eggs/pdfrw-0.1-py2.7.egg',
'/usr/lib/python2.7/dist-packages',
'.../eggs/repoze.sendmail-3.2-py2.7.egg',
'.../eggs/Beaker-1.6.4-py2.7.egg',
'.../eggs/PasteDeploy-1.5.0-py2.7.egg',
'.../eggs/translationstring-1.1-py2.7.egg',
'.../eggs/venusian-1.0a6-py2.7.egg',
'.../eggs/zope.deprecation-4.0.0-py2.7.egg',
'.../eggs/zope.interface-4.0.3-py2.7-linux-i686.egg',
'.../eggs/repoze.lru-0.6-py2.7.egg',
'.../eggs/WebOb-1.2.3-py2.7.egg',
'.../eggs/Mako-0.7.2-py2.7.egg',
'.../eggs/Chameleon-2.11-py2.7.egg',
'.../eggs/zope.sqlalchemy-0.7.1-py2.7.egg',
'.../eggs/waitress-0.8.1-py2.7.egg',
'.../eggs/transaction-1.3.0-py2.7.egg',
'.../eggs/pyramid_tm-0.5-py2.7.egg',
'.../eggs/MarkupSafe-0.15-py2.7-linux-i686.egg',
]
import cpi.scripts.initializedb
if __name__ == '__main__':
sys.exit(cpi.scripts.initializedb.main())
You may notice the “/usr/lib/python2.7/dist-packages” right in the
middle of the generated load path::
$ bin/python
>>> import zope.interface
>>> print(zope.interface)
On Mon, Feb 11, 2013 at 1:01 PM, Lele Gaifax
I do understand that the documentation says “use virtualenv†to fix the issue,
Or a clean Python build.
but I'm curious about the position of the “dist-packages†in the list:
1) why does it ends in the middle?
The paths are listed in the order that the corresponding distributions are found when satisfying requirements. So paths corresponding to the requirements you list will tend to appear earlier than the paths corresponding to their requirements, and so on. In some system installs, many system-installed distributions end up in the same path.
2) is (the position) controllable/explicitly determinable in some way?
Not directly. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
Jim Fulton
but I'm curious about the position of the “dist-packages†in the list:
1) why does it ends in the middle?
The paths are listed in the order that the corresponding distributions are found when satisfying requirements. So paths corresponding to the requirements you list will tend to appear earlier than the paths corresponding to their requirements, and so on.
In some system installs, many system-installed distributions end up in the same path.
But that's a reason why the site-packages should be moved to the end of the paths that are prepended to sys.path: When pinning a package to a specific version, it may end up *not* being used, because site-packages has the same package with a different version. If site-packages is at the end, that problem goes away. Can this be implemented or is there a technical reason for not doing it? -- Cheers Ralf
On Fri, Feb 22, 2013 at 11:11 AM, Ralf Schmitt
Jim Fulton
writes: but I'm curious about the position of the “dist-packages†in the list:
1) why does it ends in the middle?
The paths are listed in the order that the corresponding distributions are found when satisfying requirements. So paths corresponding to the requirements you list will tend to appear earlier than the paths corresponding to their requirements, and so on.
In some system installs, many system-installed distributions end up in the same path.
But that's a reason why the site-packages should be moved to the end of the paths that are prepended to sys.path:
When pinning a package to a specific version, it may end up *not* being used, because site-packages has the same package with a different version. If site-packages is at the end, that problem goes away.
Can this be implemented or is there a technical reason for not doing it?
It can't be implemented in any sane way. The problem is that site-packages doesn't have a predictable location, as demonstrated by the example given in this thread. On some systems, there are multiple directories containing custom modules. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
Jim Fulton
The problem is that site-packages doesn't have a predictable location, as demonstrated by the example given in this thread. On some systems, there are multiple directories containing custom modules.
Sorry to insist (being dumb :), but couldn't buildout, that should know the list of explicitly requested distributions, sort the paths giving precedence to them? thank you, bye, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. lele@metapensiero.it | -- Fortunato Depero, 1929.
Lele Gaifax
Jim Fulton
writes: The problem is that site-packages doesn't have a predictable location, as demonstrated by the example given in this thread. On some systems, there are multiple directories containing custom modules.
Sorry to insist (being dumb :), but couldn't buildout, that should know the list of explicitly requested distributions, sort the paths giving precedence to them?
The following uses working_set.resolve to do that. I may miss some details here, but it looks like this can be done in a sane way... diff --git a/src/zc/buildout/easy_install.py b/src/zc/buildout/easy_install.py index 75f7047..48acda0 100644 --- a/src/zc/buildout/easy_install.py +++ b/src/zc/buildout/easy_install.py @@ -889,7 +889,14 @@ def scripts(reqs, working_set, executable, dest=None, ): assert executable == sys.executable, (executable, sys.executable) - path = [dist.location for dist in working_set] + if isinstance(reqs, str): + raise TypeError('Expected iterable of requirements or entry points,' + ' got string.') + + parsed_requirements = [pkg_resources.Requirement.parse(req) for req in reqs if isinstance(req, str)] + path = [dist.location for dist in working_set.resolve(parsed_requirements)] + path.extend([dist.location for dist in working_set]) + path.extend(extra_paths) # order preserving unique unique_path = [] @@ -900,9 +907,6 @@ def scripts(reqs, working_set, executable, dest=None, generated = [] - if isinstance(reqs, str): - raise TypeError('Expected iterable of requirements or entry points,' - ' got string.') if initialization: initialization = '\n'+initialization+'\n'
Ralf Schmitt
The following uses working_set.resolve to do that. I may miss some details here, but it looks like this can be done in a sane way...
well, it shuffles the list of paths quite a bit. given the fact that you try to keep it in the same order it might not be that good or even just plain wrong.
On Fri, Feb 22, 2013 at 1:29 PM, Lele Gaifax
Jim Fulton
writes: The problem is that site-packages doesn't have a predictable location, as demonstrated by the example given in this thread. On some systems, there are multiple directories containing custom modules.
Sorry to insist (being dumb :), but couldn't buildout, that should know the list of explicitly requested distributions, sort the paths giving precedence to them?
It *could* sort them by the order it encountered them when it was computing the working set, however, if one of the requirements it finds is in a system directory, and it happens to come early, the system directory can have other distributions that shadow distributions found by buildout. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On Sat, Feb 23, 2013 at 09:54:44AM -0500, Jim Fulton wrote:
On Fri, Feb 22, 2013 at 1:29 PM, Lele Gaifax
wrote: Jim Fulton
writes: The problem is that site-packages doesn't have a predictable location, as demonstrated by the example given in this thread. On some systems, there are multiple directories containing custom modules.
Sorry to insist (being dumb :), but couldn't buildout, that should know the list of explicitly requested distributions, sort the paths giving precedence to them?
It *could* sort them by the order it encountered them when it was computing the working set, however, if one of the requirements it finds is in a system directory, and it happens to come early, the system directory can have other distributions that shadow distributions found by buildout.
And since distribute is usually in dist-packages on Ubuntu, and distribute is the first requirement needed for buildbot itself, I always get dist-packages as first item in sys.path in bin/buildout. And then all my buildouts fail because Ubuntu ships zope.interface 3.6.1 in dist-packages, and it is treated as a develop egg. Marius Gedminas -- "I'll be Bach." -- Johann Sebastian Schwarzenegger
On 23-02-13 19:13, Marius Gedminas wrote:
It*could* sort them by the order it encountered them when it was computing the working set, however, if one of the requirements it finds is in a system directory, and it happens to come early, the system directory can have other distributions that shadow distributions found by buildout. And since distribute is usually in dist-packages on Ubuntu, and distribute is the first requirement needed for buildbot itself, I always get dist-packages as first item in sys.path in bin/buildout.
And then all my buildouts fail because Ubuntu ships zope.interface 3.6.1 in dist-packages, and it is treated as a develop egg.
I'm missing something. Bootstrap and buildout grab distributions off pypi for what it needs. All that stuff ends up in the "eggs/" directory. So where/how exactly does it look for system packages? My bin/buildout normally looks something like this: #!/usr/bin/python import sys sys.path[0:0] = [ '/Users/reinout/Dotfiles/buildout/eggs/zc.buildout-2.0.1-py2.7.egg', '/Users/reinout/Dotfiles/buildout/eggs/distribute-0.6.34-py2.7.egg', ] import zc.buildout.buildout if __name__ == '__main__': sys.exit(zc.buildout.buildout.main()) The only way I know to get buildout to search the system for packages (instead of grabbing them themselves) is to use a special recipe like https://pypi.python.org/pypi/syseggrecipe What am I missing? Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
Reinout van Rees
I'm missing something. Bootstrap and buildout grab distributions off pypi for what it needs. All that stuff ends up in the "eggs/" directory. So where/how exactly does it look for system packages?
My bin/buildout normally looks something like this:
Yes, mine too. But I started the thread talking about scripts *installed* by bin/buildout, not about itself If instead the question was actually "what injects in the scripts' list of required distributions the system packages global directory".
The only way I know to get buildout to search the system for packages (instead of grabbing them themselves) is to use a special recipe like https://pypi.python.org/pypi/syseggrecipe
What am I missing?
Uhm, never used syseggrecipe actually, only zc.recipe.egg. I wonder if the two should be merged: could it be possible for zc.recipe.egg grow a new "filter-out-these-paths", and maybe a "prepend-these-paths" options for a better control of the generated scripts "sys.path"? ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. lele@metapensiero.it | -- Fortunato Depero, 1929.
participants (11)
-
Adam GROSZER
-
Chris McDonough
-
Chris Withers
-
Jim Fulton
-
Lele Gaifax
-
Lennart Regebro
-
Marius Gedminas
-
Mathieu Leduc-Hamel
-
Ralf Schmitt
-
Reinout van Rees
-
Sebastien Douche