Mailman 3 December 2011 - Distutils-SIG

Two fixes (at last)
by Greg Ward 29 Apr '18

29 Apr '18

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker 29 Apr '18

29 Apr '18

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller 29 Apr '18

29 Apr '18

3 2

[buildout] branches languishing? (site-packages and distutils scripts)
by Chris Withers 02 Apr '12

02 Apr '12

Hi All, I'm wondering what the state of play is with the following branches: reinout-scripts gary-4-include-site-packages What more needs to happen for these to get merged to trunk and a release made? cheers, Chris

10 28

[buildout] latest bootstrap.py fails on all 3 pythons that ship with Mac OS X Lion
by Chris Withers 01 Jan '12

01 Jan '12

Hi All, I just upgraded a machine I want to use as a build server to Mac OS X Lion. It appears that Lion ships with Python 2.5, 2.6 and 2.7, but bootstrap.py fails with each of these: $ python2.5 bootstrap.py The required version of setuptools (>=0.6c11) is not available, and can't be installed while this script is running. Please install a more recent version first, using 'easy_install -U setuptools'. (Currently using setuptools 0.6c9 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python)) Okay, so lets try doing what we're told: $ sudo /usr/bin/easy_install-2.5 -U setuptools ... Using /Library/Python/2.5/site-packages/setuptools-0.6c12dev_r88846-py2.5.egg ... $ python2.5 ... >>> import setuptools >>> setuptools.__version__ '0.6c12' Right, so we're all good to go? $ python2.5 bootstrap.py The required version of setuptools (>=0.6c11) is not available, and ... Wha? My guess here is that the setuptools that ships with MacOS is in /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python, and so while the bootstrap does all its hard work to get rid of site-packages, all it's actually doing is getting rid of the setuptools it can work with. Any ideas on what needs to be done to fix the bootstrap? Now, the failures in python2.6 and 2.7 are similar and all the more bizarre: $ python2.6 bootstrap.py Traceback (most recent call last): File "bootstrap.py", line 159, in <module> import pkg_resources ... File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plistlib.py", line 436, in getData LookupError: unknown encoding: ascii Which is odd, since importing pkg_resources outside of the bootstrap works fine: $ python2.6 Python 2.6.7 (r267:88850, Jul 31 2011, 19:30:54) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pkg_resources >>> 2.7 fails in the same way: $ python2.7 bootstrap.py Traceback (most recent call last): File "bootstrap.py", line 159, in <module> import pkg_resources ... File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plistlib.py", line 436, in getData LookupError: unknown encoding: ascii Any ideas? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

2 3

Problem Using Python 2.7 and Py2Exe with Pygame Module
by CT 31 Dec '11

31 Dec '11

Hi! I am getting errors and missing modules now that I have loaded Python2.7 and the Pygame modules, but works fine on my old machine using Python2.5 I would like to know what is going wrong here? I also looked on the web and lots of people are having the same problems. Does this mean that Python 2.7 is not wise to use along with Py2Exe when using Pygame modules? Below is the errors and at the end is my Setup file: *** copy extensions *** *** copy dlls *** copying c:\python27\lib\site-packages\py2exe\run.exe -> C:\Make\dist\MusicPlayer.exe The following modules appear to be missing ['AppKit', 'Foundation', 'Numeric', 'OpenGL.GL', '_scproxy', 'copyreg', 'dummy.Process', 'numpy', 'pkg_resources', 'queue', 'winreg', 'pygame.sdlmain_osx'] *** binary dependencies *** Your executable(s) also depend on these dlls which are not included, you may or may not need to distribute them. Make sure you have the license if you distribute any of them, and make sure you don't distribute files belonging to the operating system. OLEAUT32.dll - C:\WINDOWS\system32\OLEAUT32.dll USER32.dll - C:\WINDOWS\system32\USER32.dll SHELL32.dll - C:\WINDOWS\system32\SHELL32.dll ole32.dll - C:\WINDOWS\system32\ole32.dll WINMM.DLL - C:\WINDOWS\system32\WINMM.DLL ADVAPI32.DLL - C:\WINDOWS\system32\ADVAPI32.DLL WS2_32.DLL - C:\WINDOWS\system32\WS2_32.DLL GDI32.dll - C:\WINDOWS\system32\GDI32.dll libogg-0.dll - c:\python27\lib\site-packages\pygame\libogg-0.dll KERNEL32.dll - C:\WINDOWS\system32\KERNEL32.dll SDL_ttf.dll - c:\python27\lib\site-packages\pygame\SDL_ttf.dll The MusicPlayer.exe Has Been Made And Is Located In The Dist Directory! Setup.py File: PY_PROG = "music.py" APP_NAME = "MusicPlayer" cfg = { 'name':APP_NAME, 'version':'1.0', 'description':'', 'author':'', 'author_email':'', 'url':'', 'py2exe.target':'', # 'py2exe.icon':'icon.ico', #64x64 'py2exe.binary':APP_NAME, #leave off the .exe, it will be added 'py2app.target':'', 'py2app.icon':'icon.icns', #128x128 'cx_freeze.cmd':'~/src/cx_Freeze-3.0.3/FreezePython', 'cx_freeze.target':'', 'cx_freeze.binary':APP_NAME, } # usage: python setup.py command # # sdist - build a source dist # py2exe - build an exe # py2app - build an app # cx_freeze - build a linux binary (not implemented) # # the goods are placed in the dist dir for you to .zip up or whatever... from distutils.core import setup, Extension try: import py2exe except: pass import sys import glob import os import shutil try: cmd = sys.argv[1] except IndexError: print 'Usage: setup.py py2exe|py2app|cx_freeze' raise SystemExit # utility for adding subdirectories def add_files( dest, generator): for dirpath, dirnames, filenames in generator: for name in 'CVS', '.svn': if name in dirnames: dirnames.remove(name) for name in filenames: if '~' in name: continue suffix = os.path.splitext(name)[1] if suffix in ('.pyc', '.pyo'): continue if name[0] == '.': continue filename = os.path.join(dirpath, name) dest.append(filename) # define what is our data data = [] add_files( data, os.walk('data')) data.extend( glob.glob('*.txt')) # define what is our source src = [] add_files( src, os.walk('lib')) src.extend( glob.glob('*.py')) # build the sdist target if cmd == 'sdist': f = open( "MANIFEST.in", "w") for l in data: f.write("include "+l+"\n") for l in src: f.write("include "+l+"\n") f.close() setup( name=cfg['name'], version=cfg['version'], description=cfg['description'], author=cfg['author'], author_email=cfg['author_email'], url=cfg['url'], ) # build the py2exe target if cmd in ('py2exe',): dist_dir = os.path.join('dist',cfg['py2exe.target']) data_dir = dist_dir src = PY_PROG dest = cfg['py2exe.binary']+'.py' shutil.copy(src,dest) setup( options={'py2exe':{ 'dist_dir':dist_dir, 'dll_excludes':['_dotblas.pyd','_numpy.pyd'] }}, # windows=[{ console=[{ 'script':dest, # 'icon_resources':[(1,cfg['py2exe.icon'])], }], ) # build the py2app target if cmd == 'py2app': dist_dir = os.path.join('dist',cfg['py2app.target']+'.app') data_dir = os.path.join(dist_dir,'Contents','Resources') from setuptools import setup src = PY_PROG dest = cfg['py2app.target']+'.py' shutil.copy(src,dest) APP = [dest] DATA_FILES = [] OPTIONS = {'argv_emulation': True, 'iconfile':cfg['py2app.icon']} setup( app=APP, data_files=DATA_FILES, options={'py2app': OPTIONS}, setup_requires=['py2app'], ) # make the cx_freeze target if cmd == 'cx_freeze': dist_dir = os.path.join('dist',cfg['cx_freeze.target']) data_dir = dist_dir os.system('%s --install-dir %s --target-name %s %s' % (cfg['cx_freeze.cmd'], cfg['cx_freeze.binary'], dist_dir, PY_PROG)) # recursively make a bunch of folders def make_dirs(dname_): parts = list(os.path.split(dname_)) dname = None while len(parts): if dname == None: dname = parts.pop(0) else: dname = os.path.join(dname,parts.pop(0)) if not os.path.isdir(dname): os.mkdir(dname) # copy data into the binaries if cmd in ('py2exe','cx_freeze','py2app'): dest = data_dir for fname in data: dname = os.path.join(dest,os.path.dirname(fname)) make_dirs(dname) if not os.path.isdir(fname): shutil.copy(fname,dname)

2 1

Best practices to package a desktop application
by Jonathan Ballet 29 Dec '11

29 Dec '11

Hi, I hav a desktop application which currently runs only on Linux using pyGtk, and is packaged using setuptools/distribute. It ships with a number of non-Python files: * documentation (README, Changelog, Authors, etc.) * .desktop file * application specific's icons * one man file * translation files * and one executable script The setup.py file looks like this so far: setup( name='myapp', ... packages=["myapp"], package_dir={"myapp": "myapp/"}, data_files=[ ('share/myapp', ['README', 'Changelog', 'Authors']), ('share/applications', ['myapp.desktop']), ('share/pixmaps', glob.glob('myapp/pixmaps/*')), ('share/man/man1', ['myapp.1']), ('share/locale/fr/LC_MESSAGES', ['mo/fr/myapp.mo']), ... numerous other translation files ... ], entry_points={'console_scripts': ['myapp=myapp:run']}, ) but I somehow have a bad feeling about this, except for the entry point (which does a great job). When I run ``python setup.py install --prefix="local"``, everything gets installed into ``site-packages/myapp/``, Python files and data files, which seems to be a feature of setuptools/distribute over plain Distutils, whereas it seems to be "cleaner" to split things as described by the ``data_files`` setting (and AFAIK, Distutils does just that). To be clear, I end up with install/lib/python2.7/site-packages/myapp.egg/myapp.py ... install/lib/python2.7/site-packages/myapp.egg/share/myapp/README ... install/lib/python2.7/site-packages/myapp.egg/share/pixmaps/myapp.png ... install/lib/python2.7/site-packages/myapp.egg/share/locale/fr/LC_MESSAGES/myapp.mo ... Whereas I "think" I would like to have this instead, which looks more organized to me: install/lib/python2.7/site-packages/myapp.egg/myapp.py ... install/share/myapp/README ... install/share/pixmaps/myapp.png ... install/share/locale/fr/LC_MESSAGES/myapp.mo ... Actually, someone reported this "bug" to me, as he was expecting the latter whereas he got the former. Also, the application currently uses a custom method to find the location of those data files (mainly the icons which are used at run-time). It does a "terrible" job of trying several well-known locations until it either reaches the file or fails with a laconic error. I guess using ``pkg_resources`` data-access API would be much better, right? Finally, I'm also concerned about Linux distribution packagers, and I would like to package the application so that it also useful and easy for them to repackage and distribute it. After reading setuptools/distribute and distutils documentations, I'm still not what is the best way to proceed. Any hints would be greatly appreciated. Regards, Jonathan

2 1

parallelize setup.py develop/build
by Andrea Crotti 18 Dec '11

18 Dec '11

Since I have to run many setup.py develop/build I wanted to try to parallelize the calls. The normal setup.py is something like from setuptools import setup setup(...) But I was wondering, why isn't it from setuptools import setup if __name__ == '__main__': setup(...) In this way it would be possible to import the setup file without actually running any code. What I don't really like in fact is to be forced to call an external subprocess call just to run some other python code.. And in that way I would have more control also on the output and the errors that I receive, does it make sense? Going back to the original question, parallelizing many build/develop calls can be actually faster or is the same in case the hard disk is still only one?

3 4

easy_install unable to install from sourceforge?
by oliver 18 Dec '11

18 Dec '11

Any idea how to get "easy_install <package-name>" to work when <package_name> is stored on SourceForge.net? I have a python package, pypubsub, that I have been hosting on SourceForge. I would rather leave the files there as I prefer the stats available, rather than storing the files on pypi. I am able to install pubsub several ways: via "python setup.py install", by downloading the egg/zip to my computer and easy_install on egg/zip, or by storing the file on pypi (tested just to see if it was an SF.net issue). So there is nothing wrong with pubsub's setup.py. But I am unable to install Pubsub using "easy_install pypubsub". This is what I get from "easy_install pypubsub" (latest setuptools, python 2.6): C:\Program Files\wxPython2.8 Docs and Demos\demo>easy_install pypubsub Searching for pypubsub Reading http://pypi.python.org/simple/pypubsub/ Reading http://pubsub.wiki.sourceforge.net Reading http://sourceforge.net/project/showfiles.php?group_id=197063 Reading http://pubsub.sourceforge.net Reading http://sourceforge.net/projects/pubsub/files/pubsub/3.1.1b1/ Reading http://downloads.sourceforge.net/pubsub Best match: PyPubSub 3.1.1b1.win32 Downloading http://sourceforge.net/projects/pubsub/files/pubsub/3.1.1b1/PyPubSub-3.1.1b… Processing download error: Couldn't find a setup script in c:\docume~1\schoenb\locals~1\temp\easy_install-sa7vdr\download Firstly the best match should be 3.1.2, available from several of those http links. Secondly easy_install complains about a missing setup script, this is bogus because it works fine directly on the file. Setuptools 0.6.11c code shows that it does extra step when installing a python dist stored on SourceForge.net that ends in "download", but given the above I don't think that bit of code works. I notice that matplotlib, also stored on sourceforge, has same issue; and that all packages I tried that are hosted on pypi can be easy_installed via "easy_install <package-name>". Any ideas how to get "easy_install pypubsub" to work properly when the egg/zips are on SourceForge.net? Oliver

3 6

dist-dir option missing in docs
by Luc Saffre 17 Dec '11

17 Dec '11

Hi, AFAICS the `dist-dir` option of the `setup.py sdist` command is not documented in http://docs.python.org/distutils/sourcedist.html. You might smile when hearing that I had to analyze the code to find out that "python setup.py --help sdist" would have given me the answer. I suggest at least simple screenshot of that command on the above page. Best regards Luc Saffre

1 0