Distutils-SIG January 2012

distutils-sig@python.org

22 participants
26 discussions

buildout: overriding the global [versions] section

by Leonardo Rochael Almeida

Hi, It seems I cannot override, for a single buildout part, the version information of the global [versions] section. Is this possible and I just overlooked something? If not, I intend to add support for that on zc.recipe.egg and/or z3c.recipe.scripts recipes. Any comments? And where can I find the source for z3c.recipe.scripts? More details below: I've frequently found myself in the situation of needing to install, on the same buildout, different parts containing eggs with conflicting version requirements. It would be useful if I could specify a version information that overrode the global [versions] information, in a part with a zc.recipe.egg or z3c.recipe.scripts recipe. Notice that specifying the version on the 'eggs' setting doesn't work to override the global [versions] specification. I.e., the following buildout.cfg: [buildout] parts = testbrower versions=versions [versions] zope.testbrowser=3.6.0a2 [testbrower] recipe = zc.recipe.egg eggs = zope.testbrowser==3.11.1 ------- Fails with: The version, 3.6.0a2, is not consistent with the requirement, 'zope.testbrowser==3.11.1'. While: Installing testbrower. Error: Bad version 3.6.0a2 What would be nice is if I could say something like: [buildout] parts = testbrower versions=versions [versions] zope.testbrowser=3.6.0a2 [testbrowser-versions] zope.testbrowser=3.11.1 [testbrower] recipe = zc.recipe.egg versions = testbrowser-versions eggs = zope.testbrowser ------- Or maybe even: (...) [testbrower] recipe = zc.recipe.egg versions = zope.testbrowser = 3.11.1 eggs = zope.testbrowser ------- And then have buildout pick my version override only while installing the eggs in that part. The buildout easy_install API can already [1] accept a local version specification in the 'versions' keyword to the zc.buildout.easy_install.install() function. It also specifies that the global version specification can be retrieved and modified through the zc.buildout.easy_install.default_versions() function. [1] http://pypi.python.org/pypi/zc.buildout/1.5.2#specifying-version-informatio… However, the docs don't say what happens if both a default_versions() is set and a "versions=" keyword is passed to .install() containing conflicting information. In a worse case scenario, I could save the results of zc.buildout.easy_install.default_versions() and restore after calling .install() with the overridden versions. Any comments, for or against? Jim? Cheers, Leo

9 years, 8 months

Compiling Cython on Windows runs into link problems because of directories containing spaces

by Almar Klein

Hi, I have a problem with compiling Cython code on Windows. I think this is a bug in distutils which is easy to solve. I have reproduced this on Python 2.6 and Python 3.2 (32 bit). The problem occurs with the native msvc compiler. Using gcc works fine. And I prefer using gcc, but sometimes you just need msvc :/ The problem is that the command to link the libraries does not put double quotes around paths that have spaces in them. Unfortunately, the path where many users have Python installed have spaces in it (e.g. c:/program files/python26). Small example: */LIBPATH:C:\Program Files (x86)\python32\libs*. Oh, and the include_dirs DO have double quotes around them. The problem is easily solved (I confirmed this) by changing msvc9compiler.py and msvccompiler.py: def library_dir_option(self, dir): # OLD VERSION return "/LIBPATH:" + dir def library_dir_option(self, dir): # FIXED VERSION if ' ' in dir and not dir.startswith('"'): dir = '"%s"' % dir return "/LIBPATH:" + dir ===== Below follows a minimal example ===== ===== test_.pyx def foo(): print('hello') ===== setup.py import os, sys from Cython.Distutils import build_ext from distutils.core import setup from distutils.extension import Extension from numpy.distutils.misc_util import get_numpy_include_dirs # Ugly hack so I can run setup.py in my IDE sys.argv = ['setup.py', 'build_ext', '--inplace'] # Init include dirs include_dirs = ['.'] include_dirs.extend(get_numpy_include_dirs()) # Creat Extensions ext_modules = [ Extension('test_', ['test_.pyx'], include_dirs=include_dirs, ), ] # Compile setup( cmdclass = {'build_ext': build_ext}, ext_modules = ext_modules, ) print('Successfully compiled cython file: test_') ===== output when running setup.py running build_ext No module named msvccompiler in numpy.distutils; trying from distutils cythoning test_.pyx to test_.c building 'test_' extension C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -I. -I"C:\Program Files (x86)\python32\lib\site-packages\numpy\core\include" -I"C:\Program Files (x86)\python32\include" -I"C:\Program Files (x86)\python32\PC" /Tctest_.c /Fobuild\temp.win32-3.2\Release\test_.obj Found executable C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO */LIBPATH:C:\Program Files (x86)\python32\libs /LIBPATH:C:\Program Files (x86)\python32\PCbuild* /EXPORT:PyInit_test_ build\temp.win32-3.2\Release\test_.obj /OUT:C:\almar\projects\py\cmu1394\test_.pyd /IMPLIB:build\temp.win32-3.2\Release\test_.lib /MANIFESTFILE:build\temp.win32-3.2\Release\test_.pyd.manifest Found executable C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe LINK : fatal error LNK1181: cannot open input file 'Files.obj'

9 years, 8 months

Re: [Distutils] [buildout] windows issue causes jenkins problems

by Chris Withers

> On 8 בינו 2012, at 15:29, Chris Withers <chris(a)simplistix.co.uk >> The weird process forking stuff that running buildout on windows >> sometimes does causes problems for Jenkins: >> >> http://jenkins.simplistix.co.uk/job/testfixtures-buildout/32/PYTHON=2.6,lab… >> >> Is there any way I can cause this not to happen? On 08/01/2012 20:31, Guy Rozendorn wrote: > Use distribute 0.6.24 Why would this make a difference? Just for confirmation, it does not: http://jenkins.simplistix.co.uk/job/testfixtures-buildout/PYTHON=2.5,label=… cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

9 years, 8 months

Creating a binary only python distribution of a C extension module and including some additional python and c files in it

by Akhi Singhania

Hi, I have a extension module in C which I want to distribute in binary format, ideally an rpm. Additionally, I want to include some python files (examples on how to use the extension module) and source for a library the module dynamically links to (c,h, and make files). How do I specify the example python file in setup.py so that it will be included in the rpm? If I specify it as scripts, I get the following error: $ python setup.py bdist --format=rpm running build_scripts creating build/scripts-2.6 error: file 'foo.py' does not exist error: Bad exit status from /var/tmp/rpm-tmp.yjws9x (%build) If I specify it as data_files, I get the following error: $ python setup.py bdist --format=rpm error: Installed (but unpackaged) file(s) found: /usr/foo.pyc /usr/foo.pyo If I specify it as py_modules, I do not get errors but it is not included in the resulting rpm. Specifying it as a script works if I use $ python setup.py bdist --format=gztar Additionally, can I control the hierarchy of how the files are laid out in the rpm? Currently, I am specifying the c,h,make files as data_files and they get placed in /usr, which is not desirable. Alternatively, do you know of a way to specify the directory hierarchy if using --format=gztar Currently, it creates a hierarchy resembling root install into /usr. If I can modify this hierarchy, I would have essentially what I want. Regards, akhi

9 years, 8 months

[buildout] latest bootstrap.py fails on all 3 pythons that ship with Mac OS X Lion

by Chris Withers

Hi All, I just upgraded a machine I want to use as a build server to Mac OS X Lion. It appears that Lion ships with Python 2.5, 2.6 and 2.7, but bootstrap.py fails with each of these: $ python2.5 bootstrap.py The required version of setuptools (>=0.6c11) is not available, and can't be installed while this script is running. Please install a more recent version first, using 'easy_install -U setuptools'. (Currently using setuptools 0.6c9 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python)) Okay, so lets try doing what we're told: $ sudo /usr/bin/easy_install-2.5 -U setuptools ... Using /Library/Python/2.5/site-packages/setuptools-0.6c12dev_r88846-py2.5.egg ... $ python2.5 ... >>> import setuptools >>> setuptools.__version__ '0.6c12' Right, so we're all good to go? $ python2.5 bootstrap.py The required version of setuptools (>=0.6c11) is not available, and ... Wha? My guess here is that the setuptools that ships with MacOS is in /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python, and so while the bootstrap does all its hard work to get rid of site-packages, all it's actually doing is getting rid of the setuptools it can work with. Any ideas on what needs to be done to fix the bootstrap? Now, the failures in python2.6 and 2.7 are similar and all the more bizarre: $ python2.6 bootstrap.py Traceback (most recent call last): File "bootstrap.py", line 159, in <module> import pkg_resources ... File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plistlib.py", line 436, in getData LookupError: unknown encoding: ascii Which is odd, since importing pkg_resources outside of the bootstrap works fine: $ python2.6 Python 2.6.7 (r267:88850, Jul 31 2011, 19:30:54) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import pkg_resources >>> 2.7 fails in the same way: $ python2.7 bootstrap.py Traceback (most recent call last): File "bootstrap.py", line 159, in <module> import pkg_resources ... File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plistlib.py", line 436, in getData LookupError: unknown encoding: ascii Any ideas? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

9 years, 8 months

Re: [Distutils] Problem Using Python 2.7 and Py2Exe with Pygame Module

by CT

Hi Again, I changed the error that happened on the compiled version which was line 26 where I was calling a import pygame.mixer. It states at that point it can ot find a DLL when I run the compiled py2exe made .exe file. So, I changed it to not import single modules and the error below happened on compile. So, no .exe was even made just by changing one line. Note: This program, PlayMusic.py or Playing.py, or any uncompiled version runs perfectly as I designed it by clicking on it or running it from the command line. The error below comes up by only importing Pygame without single imports such as Pygame.Mixer where it will compile but fails when running it, I mean running the compiled .exe file. Inside my batch file I check for the .exe file and if not created I state "No Compile" at the very end. Bruce Failure when importing Pygame without specifics: This is the last step when attempting to copy DLLS *** copy dlls *** copying c:\python27\lib\site-packages\pygame\SDL_mixer.dll -> C:\Make\dist copying C:\WINDOWS\system32\python27.dll -> C:\Make\dist setting sys.winver for 'C:\Make\dist\python27.dll' to 'py2exe' Traceback (most recent call last): File "SetUp4.py", line 107, in <module> 'script':dest, File "c:\python27\lib\distutils\core.py", line 152, in setup dist.run_commands() File "c:\python27\lib\distutils\dist.py", line 953, in run_commands self.run_command(cmd) File "c:\python27\lib\distutils\dist.py", line 972, in run_command cmd_obj.run() File "c:\python27\lib\site-packages\py2exe\build_exe.py", line 243, in run self._run() File "c:\python27\lib\site-packages\py2exe\build_exe.py", line 312, in _run self.create_binaries(py_files, extensions, dlls) File "c:\python27\lib\site-packages\py2exe\build_exe.py", line 525, in create_ binaries self.copy_dlls(dlls) File "c:\python27\lib\site-packages\py2exe\build_exe.py", line 471, in copy_dl ls self.patch_python_dll_winver(dst) File "c:\python27\lib\site-packages\py2exe\build_exe.py", line 1008, in patch_ python_dll_winver add_resource(unicode_name, mfest, RT_MANIFEST, 2, False) RuntimeError: EndUpdateResource: The system cannot open the device or file specified. Error! No Compile! Sent: Friday, December 30, 2011 8:19 AM Subject: [Distutils] Problem Using Python 2.7 and Py2Exe with Pygame Module Hi! I am getting errors and missing modules now that I have loaded Python2.7 and the Pygame modules, but works fine on my old machine using Python2.5 I would like to know what is going wrong here? I also looked on the web and lots of people are having the same problems. Does this mean that Python 2.7 is not wise to use along with Py2Exe when using Pygame modules? Below is the errors and at the end is my Setup file: *** copy extensions *** *** copy dlls *** copying c:\python27\lib\site-packages\py2exe\run.exe -> C:\Make\dist\MusicPlayer.exe The following modules appear to be missing ['AppKit', 'Foundation', 'Numeric', 'OpenGL.GL', '_scproxy', 'copyreg', 'dummy.Process', 'numpy', 'pkg_resources', 'queue', 'winreg', 'pygame.sdlmain_osx'] *** binary dependencies *** Your executable(s) also depend on these dlls which are not included, you may or may not need to distribute them. Make sure you have the license if you distribute any of them, and make sure you don't distribute files belonging to the operating system. OLEAUT32.dll - C:\WINDOWS\system32\OLEAUT32.dll USER32.dll - C:\WINDOWS\system32\USER32.dll SHELL32.dll - C:\WINDOWS\system32\SHELL32.dll ole32.dll - C:\WINDOWS\system32\ole32.dll WINMM.DLL - C:\WINDOWS\system32\WINMM.DLL ADVAPI32.DLL - C:\WINDOWS\system32\ADVAPI32.DLL WS2_32.DLL - C:\WINDOWS\system32\WS2_32.DLL GDI32.dll - C:\WINDOWS\system32\GDI32.dll libogg-0.dll - c:\python27\lib\site-packages\pygame\libogg-0.dll KERNEL32.dll - C:\WINDOWS\system32\KERNEL32.dll SDL_ttf.dll - c:\python27\lib\site-packages\pygame\SDL_ttf.dll The MusicPlayer.exe Has Been Made And Is Located In The Dist Directory! Setup.py File: PY_PROG = "music.py" APP_NAME = "MusicPlayer" cfg = { 'name':APP_NAME, 'version':'1.0', 'description':'', 'author':'', 'author_email':'', 'url':'', 'py2exe.target':'', # 'py2exe.icon':'icon.ico', #64x64 'py2exe.binary':APP_NAME, #leave off the .exe, it will be added 'py2app.target':'', 'py2app.icon':'icon.icns', #128x128 'cx_freeze.cmd':'~/src/cx_Freeze-3.0.3/FreezePython', 'cx_freeze.target':'', 'cx_freeze.binary':APP_NAME, } # usage: python setup.py command # # sdist - build a source dist # py2exe - build an exe # py2app - build an app # cx_freeze - build a linux binary (not implemented) # # the goods are placed in the dist dir for you to .zip up or whatever... from distutils.core import setup, Extension try: import py2exe except: pass import sys import glob import os import shutil try: cmd = sys.argv[1] except IndexError: print 'Usage: setup.py py2exe|py2app|cx_freeze' raise SystemExit # utility for adding subdirectories def add_files( dest, generator): for dirpath, dirnames, filenames in generator: for name in 'CVS', '.svn': if name in dirnames: dirnames.remove(name) for name in filenames: if '~' in name: continue suffix = os.path.splitext(name)[1] if suffix in ('.pyc', '.pyo'): continue if name[0] == '.': continue filename = os.path.join(dirpath, name) dest.append(filename) # define what is our data data = [] add_files( data, os.walk('data')) data.extend( glob.glob('*.txt')) # define what is our source src = [] add_files( src, os.walk('lib')) src.extend( glob.glob('*.py')) # build the sdist target if cmd == 'sdist': f = open( "MANIFEST.in", "w") for l in data: f.write("include "+l+"\n") for l in src: f.write("include "+l+"\n") f.close() setup( name=cfg['name'], version=cfg['version'], description=cfg['description'], author=cfg['author'], author_email=cfg['author_email'], url=cfg['url'], ) # build the py2exe target if cmd in ('py2exe',): dist_dir = os.path.join('dist',cfg['py2exe.target']) data_dir = dist_dir src = PY_PROG dest = cfg['py2exe.binary']+'.py' shutil.copy(src,dest) setup( options={'py2exe':{ 'dist_dir':dist_dir, 'dll_excludes':['_dotblas.pyd','_numpy.pyd'] }}, # windows=[{ console=[{ 'script':dest, # 'icon_resources':[(1,cfg['py2exe.icon'])], }], ) # build the py2app target if cmd == 'py2app': dist_dir = os.path.join('dist',cfg['py2app.target']+'.app') data_dir = os.path.join(dist_dir,'Contents','Resources') from setuptools import setup src = PY_PROG dest = cfg['py2app.target']+'.py' shutil.copy(src,dest) APP = [dest] DATA_FILES = [] OPTIONS = {'argv_emulation': True, 'iconfile':cfg['py2app.icon']} setup( app=APP, data_files=DATA_FILES, options={'py2app': OPTIONS}, setup_requires=['py2app'], ) # make the cx_freeze target if cmd == 'cx_freeze': dist_dir = os.path.join('dist',cfg['cx_freeze.target']) data_dir = dist_dir os.system('%s --install-dir %s --target-name %s %s' % (cfg['cx_freeze.cmd'], cfg['cx_freeze.binary'], dist_dir, PY_PROG)) # recursively make a bunch of folders def make_dirs(dname_): parts = list(os.path.split(dname_)) dname = None while len(parts): if dname == None: dname = parts.pop(0) else: dname = os.path.join(dname,parts.pop(0)) if not os.path.isdir(dname): os.mkdir(dname) # copy data into the binaries if cmd in ('py2exe','cx_freeze','py2app'): dest = data_dir for fname in data: dname = os.path.join(dest,os.path.dirname(fname)) make_dirs(dname) if not os.path.isdir(fname): shutil.copy(fname,dname) _______________________________________________ Distutils-SIG maillist - Distutils-SIG(a)python.org http://mail.python.org/mailman/listinfo/distutils-sig

9 years, 8 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG January 2012