Mailman 3 August 2010 - Distutils-SIG

Two fixes (at last)
by Greg Ward 29 Apr '18

29 Apr '18

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker 29 Apr '18

29 Apr '18

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller 29 Apr '18

29 Apr '18

3 2

[buildout] branches languishing? (site-packages and distutils scripts)
by Chris Withers 02 Apr '12

02 Apr '12

Hi All, I'm wondering what the state of play is with the following branches: reinout-scripts gary-4-include-site-packages What more needs to happen for these to get merged to trunk and a release made? cheers, Chris

10 28

Buildout release news
by Gary Poster 12 Sep '10

12 Sep '10

I intend to release zc.buildout 1.5.0 Monday, August 23. I will merge the code from from svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/gary-betafix into trunk. This will change the bootstrap.py script on which some people depend via svn externals. Also, existing bootstrap and buildout invocations that do not specify a particular buildout release will automatically get the new release, as usual. In light of the troubles with the 1.5.0b1 release, particularly for users of virtualenv, I have done four things. 1. I added an automated test for the virtualenv interaction problem encountered in the last release, and fixed it. If you are using virtualenv, it now uses the old behavior for generating scripts. I and others have had success manually testing virtualenv + buildout with with these changes. 2. Today I released zc.buildout 1.4.4 (http://pypi.python.org/pypi/zc.buildout/1.4.4). The 1.4.4 release is a release for people who encounter trouble with the 1.5 line. By switching to the associated bootstrap script (http://svn.zope.org/repos/main/zc.buildout/branches/1.4/bootstrap/bootstrap…) you can stay on 1.4.4 until you are ready to migrate. 3. I have a new help document describing the changes in the 1.5 line. This will be part of the documentation shown on pypi. You can see it here: http://svn.zope.org/zc.buildout/branches/gary-betafix/SYSTEM_PYTHON_HELP.tx… 4. For the future, the new (1.5) versions of Buildout and the bootstrap script now prefer final versions of Buildout, recipes, and extensions. This can be changed by using the --accept-buildout-test-releases flag (or -t for short) when calling bootstrap. This will hopefully allow beta releases of these items to be more easily and safely made in the future. The change log is here: http://svn.zope.org/zc.buildout/branches/gary-betafix/CHANGES.txt?view=mark… Gary

11 44

any way to stop buildout/setuptools compiling things that aren't python?
by Chris Withers 30 Aug '10

30 Aug '10

Hi All, Some packages include things which aren't python, they may even have .py extensions though. Installing these using easy_install/buildout/pip/etc results in this kind of (harmless) ugliness: SyntaxError: ("'return' outside function", ('build/bdist.macosx-10.3-fat/egg/Products/ZSyncer/skins/zsyncer_skins/get_zsyncer_status_for_objects.py', 21, None, 'return statuses\n')) How can I make that not happen? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

2 4

zope.testrunner behavior change
by Barry Warsaw 30 Aug '10

30 Aug '10

I'm just back from a mini-vacation. Before I left, I upgrade the Mailman 3 branch to use buildout 1.5, but I experienced a build failure [1] and a bunch of test failures. Gary fixed all the build problems (thanks!) and I think released a new zope.testrunner package that got my branch back into working state. Mostly. I had one test failure that was fairly easy to fix, but it pointed to a change in zope.testrunner. This may be a question better suited to the TIP list, but well, I'll send it here as a follow up to my buildout issues. ;) The diff should illustrate the behavior change in z.t that broke the test: % bzr diff -c 6934 === modified file 'src/mailman/app/docs/hooks.txt' --- src/mailman/app/docs/hooks.txt 2010-01-12 13:27:38 +0000 +++ src/mailman/app/docs/hooks.txt 2010-08-30 01:00:26 +0000 @@ -51,7 +51,8 @@ >>> def call(): ... proc = subprocess.Popen( ... 'bin/mailman lists --domain ignore -q'.split(), - ... cwd='../..', # testrunner runs from ./parts/test + ... # testrunner runs from ./parts/test/working-directory + ... cwd='../../..', ... env=dict(MAILMAN_CONFIG_FILE=config_path, ... PYTHONPATH=config_directory), ... stdout=subprocess.PIPE, stderr=subprocess.PIPE) The test is trying to run a script that gets built into the bin directory of the built-out tree. I had to set cwd in order to find the bin directory. I probably could have used other landmarks to more dynamically find the bin directory, but this seemed expedient... until z.t changed the cwd in the new version. So my question is: would it make sense to be able to ask zope.testrunner where it's running from, so that I don't have to hard code cwd? Better still would be a function in z.t that I could call that would get me back to root directory. E.g. something like: ... cwd=zope.testrunner.root_directory() Thoughts? It's not that big of a deal, but I don't like that my test relies on undocumented zope.testrunner behavior. Cheers, -Barry [1] https://bugs.edge.launchpad.net/zc.buildout/+bug/623590

2 2

zc.buildout complains about site-dirs option in distutils.cfg ?
by adso.lists 28 Aug '10

28 Aug '10

Hi all, I am compiling python 2.4.6 from sources to build my custom python environment in my home directory and i set the next files in the enunciate locations to make sure that the sys.path is modified ( even i am using the standard site-package directory by now ) Regards, /home/user/opt/lib/python2.4/distutils/distutils.cfg , /home/user/opt/lib/python2.4/site-packages/sitecustomize.py /home/user/opt/lib/python2.4/site-packages/altinstall.pth /home/user/opt/lib/python2.4/site-packages/myapp.pth The point is that, when i run buildout, i get an error from zc.buildout about site-dir is not in sys.path , but if i execute the python interpreter that i have just build i can see that "/home/user/opt/lib/python2.4/site-packages" it is in sys.path even more than once . Commenting site-dir solves the issue. Any clue ? Thanks! __________________________error_____________________________________________ Getting distribution for 'zc.buildout==1.1.1'. error: /home/user/opt/lib/python2.4/site-packages (in --site-dirs) is not on sys.path An error occurred when trying to install zc.buildout 1.1.1. Look above this message for any errors that were output by easy_install. While: Bootstrapping. Getting distribution for 'zc.buildout==1.1.1'. Error: Couldn't install: zc.buildout 1.1.1 user@host:~/utilities/zish-0.2/build$ python Python 2.4.6 (#4, Aug 28 2010, 14:47:55) [GCC 4.3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', '/home/user/opt/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/ZopeSkel-2.17-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/Cheetah-2.2.1-py2.4-linux-i686.egg', '/home/user/opt/lib/python2.4/site-packages/PasteScript-1.7.3-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/PasteDeploy-1.3.3-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/Paste-1.7.4-py2.4.egg', '/home/user/opt/lib/python2.4/site-packages/virtualenv-1.4.9-py2.4.egg', '/home/user/opt/lib/python24.zip', '/home/user/opt/lib/python2.4', '/home/user/opt/lib/python2.4/plat-linux2', '/home/user/opt/lib/python2.4/lib-tk', '/home/user/opt/lib/python2.4/lib-dynload', '/home/user/opt/lib/python2.4/site-packages', '/home/user/opt/lib/python2.4/site-packages/PIL', '/home/user/opt/lib/python2.4/site-packages'] >>> #--------------------------distutils.cfg ------------------------------------- [build] compiler = unix [install] [easy_install] install-dir = /home/user/opt/lib/python2.4/site-packages script-dir = /home/user/opt/bin build-directory = /home/user/opt/tmp # uncommented raise an error: #error: /home/user/opt/lib/python2.4/site-packages (in --site-dirs) is not on sys.path #site-dirs = /home/user/opt/lib/python2.4/site-packages #--------------------------sitecustomize.py ------------------------------------- #site.addsitedir(sitedir, known_paths=None) #Adds a directory to sys.path and processes its pth files. try: import os import sys import site except ImportError: pass else: sys.setdefaultencoding("utf-8") site.addsitedir(os.path.abspath("/home/user/opt/lib/python2.4/site-packages")) #-------------------------- altinstall.pth ------------------------------------- import os, site site.addsitedir(os.path.abspath("/home/user/opt/lib/python2.4/site-packages")) #-------------------------- myapp.pth ------------------------------------- # quite redundant righ ? /home/user/opt/lib/python2.4 /home/user/opt/lib/python2.4/site-packages

1 0

buildbot for buildout (again)
by Adam GROSZER 28 Aug '10

28 Aug '10

Hello, After the 1.5.0 I tried again to setup a buildbot (and on top of that on *windows*) for buildout. Good news is that the tests now work. Bad news is that there are a hell lot of failures. More bad news is that there are some failures even on linux (a local ubuntu). Does anyone care about having a buildbot for buildout and fixing those failures? -- Best regards, Adam GROSZER mailto:agroszer@gmail.com -- Quote of the day: Kiss your keyboard goodbye!

4 5

[issue113] namespace packages installed with --single-version-externally-managed break other packages in that namespace
by Carl Meyer 27 Aug '10

27 Aug '10

New submission from Carl Meyer <carl(a)oddbird.net>: When a namespace package is installed using the --single-version-externally-managed flag, setuptools generates a special nspkg .pth file for that package to make it work as a namespace package (instead of calling pkg_resources.declare_namespace in the init.py file). When a namespace package using this .pth file is present in site-packages, any other packages installed in that same namespace that use the declare_namespace method will not be available for import. The magic code in the nspkg .pth file should be fixed to make it work with packages in the same namespace using declare_namespace. To reproduce this issue in a virtualenv, try the following: $ virtualenv --no-site-packages testenv # this installs zope.interface as an egg, with declare_namespace in zope/__init__.py $ testenv/bin/easy_install zope.interface $ testenv/bin/python -c "import zope.interface" # this installs zope.event using --single-version-externally-managed - pip uses this flag for all installations $ testenv/bin/pip install zope.event $ testenv/bin/python -c "import zope.interface" Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: No module named interface The use of easy_install and pip above is for convenience and not necessary to reproduce the bug. Downloading the sdists for zope.interface and zope.event, and installing them with "python setup.py install" (and --single-version-externally-managed for zope.event) will show the same symptom. I can also reproduce the same issue without a virtualenv on an Ubuntu Karmic system that has zope.interface installed as a system package (which uses declare_namespace in its init.py). If I install zope.event using --single-version-externally-managed, the system zope.interface is no longer importable. ---------- messages: 537 nosy: carljm priority: bug status: unread title: namespace packages installed with --single-version-externally-managed break other packages in that namespace _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue113> _______________________________________________

1 0