Mailman 3 January 2012 - Distutils-SIG

Two fixes (at last)
by Greg Ward April 29, 2018

April 29, 2018

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker April 29, 2018

April 29, 2018

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller April 29, 2018

April 29, 2018

3 2

[buildout] branches languishing? (site-packages and distutils scripts)
by Chris Withers April 2, 2012

April 2, 2012

Hi All, I'm wondering what the state of play is with the following branches: reinout-scripts gary-4-include-site-packages What more needs to happen for these to get merged to trunk and a release made? cheers, Chris

10 28

Is the Packaging Guide dead?
by anatoly techtonik Feb. 5, 2012

Feb. 5, 2012

The last commit to Packaging Guide was 2010-07-06. https://bitbucket.org/tarek/hitchhiker-guide-packaging/ The last message in the mailing list was 2010-05-05 https://groups.google.com/forum/#!forum/packaging-guide Is the project dead or moved? http://guide.python-distribute.org/ -- anatoly t.

3 4

buildout: several fold performance increases
by Ross Patterson Jan. 29, 2012

Jan. 29, 2012

I've long been perplexed by how long a buildout takes to run with multiple parts whose required distributions are largely similar. Taking a stab at it, I found two hot spots that yield several fold improvements in performance. First, zc.buildout.easy_install._log_requirements was doing expensive requirements parsing and sorting even when no message would be logged. I committed a fix for it that on a 10 part buildout with a large "eggs" option for each part decreased update time from a cProfile run time of 93 seconds to 15 seconds: http://svn.zope.org/zc.buildout/trunk/src/zc/buildout/easy_install.py?rev=1… Secondly, instantiating pkg_resources.Environment, including the setuptools.package_index.PackageIndex subclass, is very expensive and was being done multiple times for any given part, and was being done for parts whose environments were identical. There was some existing global caching for package indexes that I've duplicated for environments in the attached patch. Unfortunately, I haven't been able to get a clean test environment for the life of me. I'm using a clean Python 2.7 build from source, turning everything in ~/.buildout/default.cfg off, and running tests in a clean checkout of the zc.buildout/trunk buildout. Even under those conditions I get 17 failing tests before any changes. With this environments cache, I see 41 failures, but I can't make sense of it. This patch yields another 2-3 fold decrease to 6 seconds for the same buildout and is driven by profiling data, not guessing. Can someone help me get this patch in? Finally, it would be great to see releases of zc.buildout with these performance improvements get out in the world. I've been hearing more and more complaints about buildout run times and these are easy fixes. If we can get the second, attached patch in quickly, then I'd say we should release with both. If not, then it's still worth it to cut a release for the first, already committed patch, which yields the greatest improvement. Thanks! Ross

6 22

buildout: referencing sub-project with bad directory form
by John Calcote Jan. 26, 2012

Jan. 26, 2012

Hi, I'm new to buildout. The project I'm working on uses pywbem (http://pywbem.sf.net), which seems to have some bad form in its project directory structure. Since pywbem isn't up on PyPI, I've checked out the project from sf.net as a subdirectory of my project. Here's a tree listing of my project directory (the basic structure was built by paster using the modern_project_template): c:\...\my-project>tree /A . +---pywbem | +---irecv | \---testsuite | \---testmofs \---src +---myproject \---my_project.egg-info Note that the pywbem sub-directory has no src directory. Its setup.py looks like this: from distutils.core import setup, Extension import sys, string, os import shutil import mof_compiler mof_compiler._build() args = {'name': 'pywbem', 'author': 'Tim Potter', 'author_email': 'tpot(a)hp.com', 'description': 'Python WBEM client library', 'long_description': __doc__, 'platforms': ['any'], 'url': 'http://pywbem.sf.net/', 'version': '0.7.0', 'license': 'LGPLv2', 'packages': ['pywbem'], # Make packages in root dir appear in pywbem module 'package_dir': {'pywbem': ''}, # Make extensions in root dir appear in pywbem module 'ext_package': 'pywbem', } setup(**args) The problem I'm having is that when I add pywbem as a devel directory to my buildout.cfg file, the sys.path is configured to assume that pywbem has a src directory so it can't find the module because '.' Is not in the sys.path. Only my own 'src' directory - and 'pywbem', of course. I was about to reorganize the pywbem project a bit, but I thought I'd ask if anyone has any ideas on how to get buildout to consume pywbem in its current form. I don't own the project, and I have no developer rights to it, so I'd like to make as few changes as possible to it. Thanks in advance, John Calcote

2 1

[buildout] windows issue causes jenkins problems
by Chris Withers Jan. 24, 2012

Jan. 24, 2012

Hi All, The weird process forking stuff that running buildout on windows sometimes does causes problems for Jenkins: http://jenkins.simplistix.co.uk/job/testfixtures-buildout/32/PYTHON=2.6,lab… Is there any way I can cause this not to happen? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

2 2

Unable to execute Python executable on Windows 2000
by Chavan, Sachindeo (GE Oil & Gas) Jan. 19, 2012

Jan. 19, 2012

Hi all, I have developed a small application using python 2.5 on XP. I have the exe converted using py2exe tool. It executes in XP. I am able to create exe in windows 2000 SP4. I'm unable trying to run the exe on Windows 2000. I get the following error Traceback (most recent call last): File "UMDC.py", line 14, in <module> File "zipextimporter.pyo", line 82, in load_module File "wx\__init__.pyo", line 45, in <module> File "zipextimporter.pyo", line 82, in load_module File "wx\_core.pyo", line 4, in <module> File "zipextimporter.pyo", line 98, in load_module ImportError: MemoryLoadLibrary failed loading wx\_core_.pyd Regards, S

3 2

Distribute instead of setuptools for virtualenv?
by Christian Ştefănescu Jan. 18, 2012

Jan. 18, 2012

Hi, I'd like to know if I can create my virtualenvs with "--distribute" without any other changes, since distribute is advertised as being backwards compatible with setuptools? I'm trying to circumvent https://github.com/pypa/virtualenv/issues/202 and Distribute comes in handy. Also, if this works just fine, wouldn't it make sense to make Distribute the default for virtualenv at some point? Thanks, Chris

3 2