Mailman 3 October 2006 - Distutils-SIG

Re: [Distutils] Need help switching to setuptools
by Phillip J. Eby Oct. 19, 2006

Oct. 19, 2006

At 06:11 PM 10/18/2006, Steven Bird wrote: >Hi, > >I'm leading the development of NLTK, the Natural Language Toolkit, a >suite of Python modules for analyzing text >[http://nltk.sourceforge.net/]. I'm strugging to work out how to >switch from distutils to setuptools, and am wondering if there's >someone out there who can work with me to do the migration. > >There are several components to the distribution: >* a dozen pure python packages >* a dependency on numpy >* a dependency on elementtree for Python < 2.5 >* textbook pdfs to be installed centrally >* 200Mb of data (uncompressed) to be installed centrally The above are relatively easy; just put the data and docs inside of your package directories, add install_package_data=True and install_requires=['numpy','elementtree'] (more or less) and import everything from setuptools instead of distutils.core. >* similar amount of data to be generated at install time and stored >centrally ("models") >* directory tree to be stored to user filespace (code samples from >the textbook) These two aren't currently supported. The closest you can get is to include scripts that users can run to generate, install, copy, etc., as eggs are simply installed as-is under normal circumstances, and to a single location. >* locations of all data to be discovered at run-time by the toolkit Use the pkg_resources API calls to do this, and you're set.

1 0

Need help switching to setuptools
by Steven Bird Oct. 19, 2006

Oct. 19, 2006

Hi, I'm leading the development of NLTK, the Natural Language Toolkit, a suite of Python modules for analyzing text [http://nltk.sourceforge.net/]. I'm strugging to work out how to switch from distutils to setuptools, and am wondering if there's someone out there who can work with me to do the migration. There are several components to the distribution: * a dozen pure python packages * a dependency on numpy * a dependency on elementtree for Python < 2.5 * textbook pdfs to be installed centrally * 200Mb of data (uncompressed) to be installed centrally * similar amount of data to be generated at install time and stored centrally ("models") * directory tree to be stored to user filespace (code samples from the textbook) * locations of all data to be discovered at run-time by the toolkit * easy installation on linux, mac and windows I believe setuptools can do all these things, but I'm stalled on my way up a steep learning curve! The NLTK subversion repository is available for anonymous access here: svn co https://svn.sourceforge.net/svnroot/nltk/trunk/nltk Any help would be gratefully appreciated. Thanks, -Steven Bird

1 0

Distributing executable binaries in eggs
by Pascoe, S (Stephen) Oct. 18, 2006

Oct. 18, 2006

I am trying create eggs for a package that relies on a compiled executable that is distributed with the package. I can get the executable into the egg fine. It is declared as package_data and I set zip_safe=False to force easy_install to unpack the egg. My problem is that the executable is always extracted without execution permissions (our platform is UNIX). After some experimenting this appears to be a general problem with the zip file module not honouring execution permissions. I know permissions can be set in zip files because the InfoZip tools do it. Currently I have a hack working that tries to change the permissions on the executable when the package is imported but this won't work if the egg is installed by a different user to the importer. I am wondering if there is a way round this with setuptools. Some sort of hook to fix permissions when unpacking a non-zipsafe egg would be good. Thanks, Stephen. --- Stephen Pascoe 01235 445980 British Atmospheric Data Centre Rutherford Appleton Laboratory, CCLRC

1 0

intensively
by Evelina Higgins Oct. 17, 2006

Oct. 17, 2006

1 0

Re: [Distutils] numpy build fails in distutils code
by Jay Parlar Oct. 16, 2006

Oct. 16, 2006

> On Oct 14, 2006, at 6:27 PM, Robert Kern wrote: > Actually it is a problem in disutils, one that is AFAIK fixed in > 2.4.4c1 (and the same patch is in the 2.5 branch and on the trunk). > > What happens is that the binary release of Python for OSX is built as > a universal binary and uses compiler flags that the compiler > toolchain on OSX 10.3.9 doesn't understand. Distutils tries to remove > those flags on 10.3.9, but I forgot a number of settings that need to > be cleaned up. Perfect, that's what I suspected was happening. I'll just wait for 2.5.1 then. Thanks a lot! Jay P.

2 1

setuputils cvs based source detection?
by S Joshua Swamidass Oct. 15, 2006

Oct. 15, 2006

Here is the situation: I have a large project in one big cvs tree. i want to be able to release in differen't packages from this tree using different setup.py's. My project is organized like: ./ (root dir) setup_A.py #package = ['A'] setup_B.py #package = ['B'] A/ __init__.py moduleA.py B/ __init__.py moduleB.py The bdist command works fine with this setup. If i use the distutils.core.setup, the sdist command works correctly too. So here is the problem: using setuputils.setup, the sdist command will package the WHOLE source tree (both directories A and B) when using either of the setup_*.py. I'm quite certain this is because both A/ and B/ are under cvs source control, and setup assumes that the whole source tree is required for the sdist. How do i turn this behaivior off or develop a work around? Thanks! S Joshua Swamidass

2 1

Re: [Distutils] numpy build fails in distutils code
by Phillip J. Eby Oct. 15, 2006

Oct. 15, 2006

At 11:10 AM 10/14/2006 -0400, Jay Parlar wrote: >I've tried asking this on the numpy list (a few times), but >unfortunately no response. Since the error *appears* to be inside >distutils, I thought maybe someone here would recognize it. Nope, it's a numpy.distutils error; notice that the bottom of the traceback is all from numpy code. Distutils also has no "config" command, so that bit's coming from numpy.distutils also.

3 2

numpy build fails in distutils code
by Jay Parlar Oct. 15, 2006

Oct. 15, 2006

I've tried asking this on the numpy list (a few times), but unfortunately no response. Since the error *appears* to be inside distutils, I thought maybe someone here would recognize it. Anyway, I'm trying to build the lastest numpy RC with Python 2.5 on OS X 10.3.9, and this is the fun that occurrs: jayparlar$ python setup.py build Running from numpy source directory. F2PY Version 2_3296 blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] running build running config_fc running build_src building py_modules sources creating build creating build/src.macosx-10.3-fat-2.5 creating build/src.macosx-10.3-fat-2.5/numpy creating build/src.macosx-10.3-fat-2.5/numpy/distutils building extension "numpy.core.multiarray" sources creating build/src.macosx-10.3-fat-2.5/numpy/core Generating build/src.macosx-10.3-fat-2.5/numpy/core/config.h customize NAGFCompiler customize AbsoftFCompiler customize IbmFCompiler Could not locate executable g77 Could not locate executable f77 Could not locate executable gfortran Could not locate executable f95 customize GnuFCompiler customize Gnu95FCompiler customize G95FCompiler customize GnuFCompiler customize Gnu95FCompiler customize NAGFCompiler customize NAGFCompiler using config C compiler: gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 compile options: '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -Inumpy/core/src -Inumpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c' gcc: _configtest.c gcc: cannot specify -o with -c or -S and multiple compilations gcc: cannot specify -o with -c or -S and multiple compilations failure. removing: _configtest.c _configtest.o numpy/core/setup.py:50: DeprecationWarning: raising a string exception is deprecated raise "ERROR: Failed to test configuration" Traceback (most recent call last): File "setup.py", line 89, in <module> setup_package() File "setup.py", line 82, in setup_package configuration=configuration ) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/core.py", line 174, in setup return old_setup(**new_attr) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/build.py", line 112, in run self.run_command(cmd_name) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 87, in run self.build_sources() File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 106, in build_sources self.build_extension_sources(ext) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 212, in build_extension_sources sources = self.generate_sources(sources, ext) File "/Users/jayparlar/Desktop/numpy-1.0rc2/numpy/distutils/command/build_src.py", line 270, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 50, in generate_config_h raise "ERROR: Failed to test configuration" ERROR: Failed to test configuration Jay-Computer:~/Desktop/numpy-1.0rc2 jayparlar$ It's failing on this: result = config_cmd.try_run(tc,include_dirs=[python_include], library_dirs = default_lib_dirs) if not result: raise "ERROR: Failed to test configuration" Here, config_cmd is a distutils.command.config.try_run Does anyone recognize this error? I'm looking in the direction of the "gcc: cannot specify -o with -c or -S and multiple compilations" as the root of my problem. Some weird combination of gcc3.3, Python2.5 and OS X 10.3.9 seems to be causing it. I'd really appreciate any insight into this, but totally understand if no one has any ideas :) Jay P.

3 2

Re: [Distutils] setuputils cvs based source detection?
by Phillip J. Eby Oct. 15, 2006

Oct. 15, 2006

At 07:55 PM 10/14/2006 -0700, S Joshua Swamidass wrote: >Here is the situation: > >I have a large project in one big cvs tree. i want to be able to >release in differen't packages from this tree using different >setup.py's. My project is organized like: > >./ (root dir) >setup_A.py #package = ['A'] >setup_B.py #package = ['B'] >A/ > __init__.py > moduleA.py >B/ > __init__.py > moduleB.py > >The bdist command works fine with this setup. If i use the >distutils.core.setup, the sdist command works correctly too. So here >is the problem: using setuputils.setup, the sdist command will package >the WHOLE source tree (both directories A and B) when using either of >the setup_*.py. > >I'm quite certain this is because both A/ and B/ are under cvs source >control, and setup assumes that the whole source tree is required for >the sdist. How do i turn this behaivior off or develop a work around? setuptools doesn't provide a way to support this behavior. Even if you were to manually modify MANIFEST.in to exclude the files you didn't want, you'd have to change it every time you issued a release, and your source distributions wouldn't be distutils-standard or easy_install-compatible anyway, because you wouldn't have a *single* "setup.py" file in the distribution, and having a "setup_A.py" isn't the same thing. My suggestion would be for you to change how you're doing this, so that you have a single setup.py for each thing you want to distribute separately, or else distribute the whole thing as one giant package. If you want to minimize the amount of directory reorganization needed, one possible approach is to put a setup.py in each of A and B, and adjust the package_dir setting of setup() accordingly. There are some drawbacks to that layout, but at least you would be able to keep your package directories where they are now.

2 1

Question on distutils.sysconfig.get_config_vars( )...
by Venkat Bommakanti Oct. 14, 2006

Oct. 14, 2006

Hi, In: get_config_vars(*args): global _config_vars if _config_vars is None: ... else: ... ... even on the first invocation, it appears _config_vars has some valid data, supposedly picked up from the makefile of the build python... But can't seem to track down where its actually reading/stuffing info into _config_vars. The build system is i686 linux. That said, I don't even see the _init_posix getting invoked for it to read the makefile. In my attemp to cross compile a C extension on a i686 linux box targetting a ppc linux box, I'm hoping to intervene and have it read the config info from the corresponding target python's makefile, and not that of the build python's... I tried asking on comp.lang.python, but to no avail... Thanks much. /venkat

1 0