distutils issue - python 3.1 on windows
Hi, I'm trying to get building to work with Python 3.1 under Wine on OS X. The first thing you run into is a python distutils problem, which is fixed by replacing line 379 of cygwinccompiler.py with result = RE_VERSION.search(str(out_string)) The next thing I run into is a numpy.distutils issue (complete build output below email): File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\mingw32ccompiler.py", line 177, in link func(*args[:func.__func__.__code__.co_argcount]) AttributeError: 'function' object has no attribute '__func__' What I think that line is for is to test what kind of arguments can be passed to 'func', but I'm really not sure. A comment would have been helpful, that must be the most obscure line of Python code I've ever seen:) Does anyone have any idea what causes this error? Thanks, Ralf <... 2to3 output ...> F2PY Version 1 Running from numpy source directory.numpy\core\code_generators\generate_umath.py:117: DeprecationWarning: string.maketrans is deprecated, use bytes.maketrans instead bytes(string.ascii_uppercase, "ascii")) blas_opt_info: blas_mkl_info: libraries mkl,vml,guide not found in C:\Python31\lib libraries mkl,vml,guide not found in C:\ libraries mkl,vml,guide not found in C:\Python31\libs NOT AVAILABLE atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in C:\Python31\lib libraries ptf77blas,ptcblas,atlas not found in C:\ libraries ptf77blas,ptcblas,atlas not found in C:\Python31\libs NOT AVAILABLE atlas_blas_info: libraries f77blas,cblas,atlas not found in C:\Python31\lib libraries f77blas,cblas,atlas not found in C:\ libraries f77blas,cblas,atlas not found in C:\Python31\libs NOT AVAILABLE Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\system_info.py:1399: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) blas_info: FOUND: libraries = ['blas'] library_dirs = ['C:\\local\\lib\\yop\\nosse'] language = f77 FOUND: libraries = ['blas'] library_dirs = ['C:\\local\\lib\\yop\\nosse'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77 lapack_opt_info: lapack_mkl_info: mkl_info: libraries mkl,vml,guide not found in C:\Python31\lib libraries mkl,vml,guide not found in C:\ libraries mkl,vml,guide not found in C:\Python31\libs NOT AVAILABLE NOT AVAILABLE atlas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in C:\Python31\lib libraries lapack_atlas not found in C:\Python31\lib libraries ptf77blas,ptcblas,atlas not found in C:\ libraries lapack_atlas not found in C:\ libraries ptf77blas,ptcblas,atlas not found in C:\Python31\libs libraries lapack_atlas not found in C:\Python31\libs <class 'numpy.distutils.system_info.atlas_threads_info'> NOT AVAILABLE atlas_info: libraries f77blas,cblas,atlas not found in C:\Python31\lib libraries lapack_atlas not found in C:\Python31\lib libraries f77blas,cblas,atlas not found in C:\ libraries lapack_atlas not found in C:\ libraries f77blas,cblas,atlas not found in C:\Python31\libs libraries lapack_atlas not found in C:\Python31\libs <class 'numpy.distutils.system_info.atlas_info'> NOT AVAILABLE Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\system_info.py:1306: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) lapack_info: FOUND: libraries = ['lapack'] library_dirs = ['C:\\local\\lib\\yop\\nosse'] language = f77 FOUND: libraries = ['lapack', 'blas'] library_dirs = ['C:\\local\\lib\\yop\\nosse'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77 could not resolve pattern in '': '*.txt' non-existing path in '': 'COMPATIBILITY' non-existing path in '': 'site.cfg.example' running build running config_cc unifing config_cc, config, build_clib, build_ext, build commands --compiler options running config_fc unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options running build_src build_src building py_modules sources creating build creating build\src.win32-3.1 creating build\src.win32-3.1\numpy creating build\src.win32-3.1\numpy\distutils building library "npymath" sources gcc -dumpversion ld -v dllwrap --version customize GnuFCompiler Found executable C:\MinGW\bin\g77.exe gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found Found executable C:\MinGW\bin\g77.exe customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler using config C compiler: gcc -mno-cygwin -O2 -Wall -Wstrict-prototypes compile options: '-Inumpy\core\src\private -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\include -IC:\Python31\include -IC:\Python31\PC -c' gcc -mno-cygwin -O2 -Wall -Wstrict-prototypes -Inumpy\core\src\private -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\include -IC:\Python31\include -IC:\Python31\PC -c _configtest.c -o _configtest.o Found executable C:\MinGW\bin\gcc.exe Traceback (most recent call last): File "setup.py", line 210, in <module> setup_package() File "setup.py", line 203, in setup_package configuration=configuration ) File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\core.py", line 186, in setup return old_setup(**new_attr) File "C:\Python31\lib\distutils\core.py", line 149, in setup dist.run_commands() File "C:\Python31\lib\distutils\dist.py", line 919, in run_commands self.run_command(cmd) File "C:\Python31\lib\distutils\dist.py", line 938, in run_command cmd_obj.run() File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\command\build.py", line 37, in run old_build.run(self) File "C:\Python31\lib\distutils\command\build.py", line 128, in run self.run_command(cmd_name) File "C:\Python31\lib\distutils\cmd.py", line 315, in run_command self.distribution.run_command(command) File "C:\Python31\lib\distutils\dist.py", line 938, in run_command cmd_obj.run() File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\command\build_src.py", line 152, in run self.build_sources() File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\command\build_src.py", line 163, in build_sources self.build_library_sources(*libname_info) File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\command\build_src.py", line 298, in build_library_sources sources = self.generate_sources(sources, (lib_name, build_info)) File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\command\build_src.py", line 385, in generate_sources source = func(extension, build_dir) File "numpy\core\setup.py", line 678, in get_mathlib_info st = config_cmd.try_link('int main(void) { return 0;}') File "C:\Python31\lib\distutils\command\config.py", line 248, in try_link libraries, library_dirs, lang) File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\command\config.py", line 149, in _link libraries, library_dirs, lang)) File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\command\config.py", line 89, in _wrap_method ret = mth(*((self,)+args)) File "C:\Python31\lib\distutils\command\config.py", line 145, in _link target_lang=lang) File "C:\Python31\lib\distutils\ccompiler.py", line 808, in link_executable debug, extra_preargs, extra_postargs, None, target_lang) File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils\mingw32ccompiler.py", line 177, in link func(*args[:func.__func__.__code__.co_argcount]) AttributeError: 'function' object has no attribute '__func__' Captured Task Output: --------------------- ---> pavement.bdist_superpack Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", line 164, in _run_task return do_task() File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/Paver-1.0.1-py2.6.egg/paver/tasks.py", line 161, in do_task return func(**kw) File "pavement.py", line 213, in bdist_superpack bdist_wininst_arch(pyver, 'nosse') File "pavement.py", line 196, in bdist_wininst_arch _bdist_wininst(pyver, SITECFG[arch]) File "pavement.py", line 266, in _bdist_wininst subprocess.check_call(cmd, env=cfg_env) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py", line 488, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['wine', '/Users/rgommers/.wine/drive_c/Python31/python.exe', 'setup.py', 'build', '-c', 'mingw32', 'bdist_wininst']' returned non-zero exit status 1
Mon, 02 Aug 2010 23:48:52 +0800, Ralf Gommers wrote:
I'm trying to get building to work with Python 3.1 under Wine on OS X. The first thing you run into is a python distutils problem, which is fixed by replacing line 379 of cygwinccompiler.py with result = RE_VERSION.search(str(out_string))
That's going to bust when out_string contains stuff that does not fit in ASCII. I don't remember how this was addressed in the rest of distutils.
The next thing I run into is a numpy.distutils issue (complete build output below email):
File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils \mingw32ccompiler.py", line 177, in link func(*args[:func.__func__.__code__.co_argcount]) AttributeError: 'function' object has no attribute '__func__'
What I think that line is for is to test what kind of arguments can be passed to 'func', but I'm really not sure. A comment would have been helpful, that must be the most obscure line of Python code I've ever seen:) Does anyone have any idea what causes this error?
This is change in the code object internals in Python 3, and 2to3 doesn't seem able make the conversions automatically. I don't remember exactly how it goes, but there are other instances of this in Numpy -- need to do one thing on Python 2 and another on Python 3. Anyway, here it seems like the `inspect` module should be used -- no need to go mucking with in the code objects to just find the number of arguments for a function. Nobody has tested this part of the distutils code on Python 3, and indeed it does not have any tests, so it's not a surprise that stuff like this is left over :) -- Pauli Virtanen
On Mon, Aug 2, 2010 at 12:48, Pauli Virtanen <pav@iki.fi> wrote:
Mon, 02 Aug 2010 23:48:52 +0800, Ralf Gommers wrote:
I'm trying to get building to work with Python 3.1 under Wine on OS X. The first thing you run into is a python distutils problem, which is fixed by replacing line 379 of cygwinccompiler.py with result = RE_VERSION.search(str(out_string))
That's going to bust when out_string contains stuff that does not fit in ASCII.
I don't remember how this was addressed in the rest of distutils.
The next thing I run into is a numpy.distutils issue (complete build output below email):
File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils \mingw32ccompiler.py", line 177, in link func(*args[:func.__func__.__code__.co_argcount]) AttributeError: 'function' object has no attribute '__func__'
What I think that line is for is to test what kind of arguments can be passed to 'func', but I'm really not sure. A comment would have been helpful, that must be the most obscure line of Python code I've ever seen:) Does anyone have any idea what causes this error?
This is change in the code object internals in Python 3, and 2to3 doesn't seem able make the conversions automatically. I don't remember exactly how it goes, but there are other instances of this in Numpy -- need to do one thing on Python 2 and another on Python 3.
Anyway, here it seems like the `inspect` module should be used -- no need to go mucking with in the code objects to just find the number of arguments for a function.
Nobody has tested this part of the distutils code on Python 3, and indeed it does not have any tests, so it's not a surprise that stuff like this is left over :)
I believe we avoided the inspect module because it is quite expensive to import. It may not matter inside numpy.distutils, but be wary of "fixing" things to use inspect elsewhere. It would be worth extracting the commonly-used pieces of inspect (and hacks like this) into an internal utility module that is fast to import. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Mon, 02 Aug 2010 12:52:12 -0500, Robert Kern wrote: [clip]
I believe we avoided the inspect module because it is quite expensive to import. It may not matter inside numpy.distutils, but be wary of "fixing" things to use inspect elsewhere. It would be worth extracting the commonly-used pieces of inspect (and hacks like this) into an internal utility module that is fast to import.
We actually have `numpy.compat._inspect` and from numpy.compat import getargspec that could be used here. -- Pauli Virtanen
On 08/03/2010 02:57 AM, Pauli Virtanen wrote:
Mon, 02 Aug 2010 12:52:12 -0500, Robert Kern wrote: [clip]
I believe we avoided the inspect module because it is quite expensive to import. It may not matter inside numpy.distutils, but be wary of "fixing" things to use inspect elsewhere. It would be worth extracting the commonly-used pieces of inspect (and hacks like this) into an internal utility module that is fast to import.
We actually have `numpy.compat._inspect` and
from numpy.compat import getargspec
that could be used here.
Yep, it was added precisely for avoiding the slow import of upstream inspect, cheers, David
Mon, 02 Aug 2010 23:48:52 +0800, Ralf Gommers wrote:
I'm trying to get building to work with Python 3.1 under Wine on OS X. The first thing you run into is a python distutils problem, which is fixed by replacing line 379 of cygwinccompiler.py with result = RE_VERSION.search(str(out_string))
Ah yep, cygwinccompiler.py is in Python's distutils, I wondered for a while where the bug was :) But actually, I'm now looking at the cygwinccompiler.py source code, and to me it seems it should be ok. The RE_VERSION is a byte-regular expression, so it's not obvious why making the input a str would help?
File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils \mingw32ccompiler.py", line 177, in link func(*args[:func.__func__.__code__.co_argcount]) AttributeError: 'function' object has no attribute '__func__'
I fixed this in trunk r8595 (cherry-pick to 1.5.x if you find it works). Pauli
On Wed, Aug 4, 2010 at 6:25 AM, Pauli Virtanen <pav@iki.fi> wrote:
Mon, 02 Aug 2010 23:48:52 +0800, Ralf Gommers wrote:
I'm trying to get building to work with Python 3.1 under Wine on OS X. The first thing you run into is a python distutils problem, which is fixed by replacing line 379 of cygwinccompiler.py with result = RE_VERSION.search(str(out_string))
Ah yep, cygwinccompiler.py is in Python's distutils, I wondered for a while where the bug was :)
But actually, I'm now looking at the cygwinccompiler.py source code, and to me it seems it should be ok. The RE_VERSION is a byte-regular expression, so it's not obvious why making the input a str would help?
I haven't started using py3k yet so I'm still a bit fuzzy about bytes vs string. But it's easy to try in the interpreter:
import re RE_VERSION = re.compile('(\d+\.\d+(\.\d+)*)') cmd = "gcc -dumpversion" from subprocess import Popen, PIPE out = Popen(cmd, shell=True, stdout=PIPE).stdout out_string = out.read() out.close() result = RE_VERSION.search(out_string) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: can't use a string pattern on a bytes-like object
File "Z:\Users\rgommers\Code\numpy\build\py3k\numpy\distutils \mingw32ccompiler.py", line 177, in link func(*args[:func.__func__.__code__.co_argcount]) AttributeError: 'function' object has no attribute '__func__'
I fixed this in trunk r8595 (cherry-pick to 1.5.x if you find it works).
That works, thanks. Then there's was still one more problem, _dotblas.c was not py3k ready. With this fix I can build everything under Wine: http://github.com/rgommers/numpy/tree/build-py3k Does that look fine? Cheers, Ralf
Wed, 04 Aug 2010 23:34:15 +0800, Ralf Gommers wrote: [clip]
I haven't started using py3k yet so I'm still a bit fuzzy about bytes vs string. But it's easy to try in the interpreter:
import re RE_VERSION = re.compile('(\d+\.\d+(\.\d+)*)')
In the Python 3.1 version I have, this line reads RE_VERSION = re.compile(b'(\d+\.\d+(\.\d+)*)') which makes it a byte-regular expression. [clip]
That works, thanks. Then there's was still one more problem, _dotblas.c was not py3k ready.
Doh! How did I miss that (answer: no tests, and the import error is masked :)
With this fix I can build everything under Wine: http://github.com/rgommers/numpy/tree/build-py3k Does that look fine?
It compiles, but will probably crash on import :) You need also to return the module object, see e.g. how it's done in numpy/numarray/_capi.c I committed a fixed version. Cheers, Pauli
participants (4)
-
David -
Pauli Virtanen -
Ralf Gommers -
Robert Kern