Problems building SVN on Windows

Hi, I am trying to build numpy on Windows from the current SVN (rev. 3884 I think) and I'm having some problems. I've successfully built ATLAS and LAPACK. I've installed Python 2.5.1 from the standard windows installer. I also have a fresh install of Cygwin with all mingw development tools etc. I created a site.cfg file from the template with the following entries: ----------- [blas_opt] libraries = f77blas, cblas, atlas library_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\lib include_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\include # [lapack_opt] libraries = lapack, f77blas, cblas, atlas library_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\lib include_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\include [fftw] libraries = fftw3 library_dirs = "C:\fftw" ----------- So the first problem is this file doesn't seem to be recognised (and these library paths aren't picked up). I've tried from the cygwin shell and windows cmd and with the file next to setup.py and also in numpy/distutils. However the only paths searched are C:\,C:\Python25\Libs,C:\Python25\lib. I got around this by copying the .a files to C:\. This is a bit messy but at least it finds ATLAS and LAPACK. Then when I try to run config/build as described on the wiki I get an error about no valid fortran compiler being defined. The full output of python setup.py -v build --compiler=mingw32 is attached. I would be greatful if anyone could help me to get this working. Also if someone could provide more details on how to ensure fftw is used. The precompiled fftw library I downloaded provides libfftw3-3.dll as well as couple of others. Will this be picked up correctly by my fftw entry in site.cfg (if site.cfg was working) or should it be "libraries=fftw3-3"? Is it OK to use this DLL or do I have to build a static library myself from source. Finally, I gather the very nice scipy website/wiki is relatively new... I was wondering if you had considered a phpbb type forum for support issues such as this. I think this can be a lot more accessible for new users (such as myself)... Thanks in advance, Robin

Hi, I am keen to evaluate numpy as an alternative to MATLAB for my PhD work and possible wider use within the department. To make a fairer comparison I wanted to build it with optimised ATLAS/LAPACK etc. hence building from source. I am running into some problems though. I am using Windows XP SP2, with latest Cygwin and I'm trying to follow the instructions on the wiki. Firstly, is what I'm trying possible? On the Installing Scipy/Windows page it says MinGW gcc and g77 are best supported, but also says to build against binary Python download you need to use MSVC. From http://boodebr.org/main/python/build-windows-extensions it seems building extensions with gcc is fine (and I built PyCrypto successfully as a test). So can I do what I am trying to do (build numpy/scipy on windows using cygwin without MSVC installed) against the downloaded Python distribution? If not, I can't find any resources about building Python from source on Windows using Cygwin, so it seems like I would be completely stuck. The next problem is that although I filled in the site.cfg file as documented (below), the setup.py script doesn't seem to pick it up and doesn't look in any of the specified directories. I can get around this by putting ATLAS/LAPACK libs in C:\, but obviously this isn't very satisfactory. Also is the entry for fftw correct? I couldn't find any information about this on the wiki. ------ [blas_opt] libraries = f77blas, cblas, atlas library_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\lib include_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\include # [lapack_opt] libraries = lapack, f77blas, cblas, atlas library_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\lib include_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\include [fftw] libraries = fftw3-3 library_dirs = "C:\fftw" ----- Following this there seem to be some more problems with the setup. "python setup.py build --compiler=mingw32" fails with: File "C:\cygwin\home\mqbxfri2\numpy_trunk\numpy\distutils\fcompiler\__init__.py", line 731, in _find_existing_fcompiler c.customize(dist) AttributeError: 'NoneType' object has no attribute 'customize' c is the result of the new_fcompiler function. "python setup.py config_fc --help-fcompiler" failes with: File "c:\Python25\lib\distutils\msvccompiler.py", line 270, in initialize "version of the compiler, but it isn't installed." % self.__product) distutils.errors.DistutilsPlatformError: Python was built with Visual Studio version 7.1, and extensions need to be built with the same version of the compiler, but it isn't installed. Although as I mentioned I can successfully build extensions with gcc. Reading through the help I saw that there is a "none" fcompiler type, so using that I get a bit further: "python setup.py build --compiler=mingw32 --fcompiler=none" run initially with build directory removed gives the same NoneType error, but then running it again the build appears to start. I then run into bug #220 http://projects.scipy.org/scipy/numpy/ticket/220: numpy\core\src\multiarraymodule.c: In function `initmultiarray': numpy\core\src\multiarraymodule.c:7563: error: `NPY_ALLOW_THREADS' undeclared (first use in this function) numpy\core\src\multiarraymodule.c:7563: error: (Each undeclared identifier is reported only once numpy\core\src\multiarraymodule.c:7563: error: for each function it appears in.) I would really appreciate any help to get this working. I understand building numpy doesn't require a fortran compiler, but scipy does. I am hoping to build scipy as well, so presumably the config system needs to recognise the cygwin g77 compiler for that to work? During the config I also see the following message: don't know how to compile Fortran code on platform 'nt' with 'gnu' compiler. Supported compilers are: absoft Does this mean it isn't possible to build scipy on Windows with Cygwin compilers? If I am eventually successful I would be happy to update the wiki with some more detailed instructions based on my experiences. Finally, I can't find any discussion of the relevant merits of ATLAS vs MKL, other than the different licensing. Is it expected that MKL performs better? Which is recommended? Thanks very much, Robin

Hi, By making replacing line 807 in fcompiler/__init__.py (return None) with: from numpy.distutils.fcompiler.none import NoneFCompiler compiler = NoneFCompiler() return compiler I have been able to get a little bit further with my problems. I'm not sure if this is the correct way to do the import since I'm quite new to Python. It seems to build OK now, but there is a linking error: I'm not sure why -lmsvcr71 is included in the linker flags, since I should be using cygwin (setup.py build --compiler=mingw32). I think this is because it is not finding the fortran libraries needed. These should come with cygwin, but I'm not sure how to point to them. Also it still reports not finding a fortran compiler, although g77 is installed in cygwin. Looking through the code I'm wondering if there is some confusion between use of os.name (which is 'nt') and sys.platform (which is 'win32') - I thought perhaps that's why its reporting not Fortran compilers supported for 'nt'. Not too sure though. Again any help greatfully received. ... lots of similar undefinied references ... C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0xe): undefined reference to `_s_wsfe' C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0x29): undefined reference to `_do_fio' C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0x44): undefined reference to `_do_fio' C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0x49): undefined reference to `_e_wsfe' C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0x5d): undefined reference to `_s_stop' collect2: ld returned 1 exit status error: Command "g++ -mno-cygwin -shared build\temp.win32-2.5\Release\numpy\linal g\lapack_litemodule.o -LC:\ -Lc:\Python25\libs -Lc:\Python25\PCBuild -llapack -l f77blas -lcblas -latlas -lpython25 -lmsvcr71 -o build\lib.win32-2.5\numpy\linalg \lapack_lite.pyd" failed with exit status 1 Thanks, Robin

FWIW, I'm fairly certain that the binaries for win32 are compiled using mingw, so I'm pretty certain that it's possible. I use MSVCC myself, so I can't be of much help though. On 7/13/07, numpy-discussion@robince.ftml.net < numpy-discussion@robince.ftml.net> wrote:
Hi,
By making replacing line 807 in fcompiler/__init__.py (return None) with: from numpy.distutils.fcompiler.none import NoneFCompiler compiler = NoneFCompiler() return compiler
I have been able to get a little bit further with my problems. I'm not sure if this is the correct way to do the import since I'm quite new to Python.
It seems to build OK now, but there is a linking error: I'm not sure why -lmsvcr71 is included in the linker flags, since I should be using cygwin (setup.py build --compiler=mingw32). I think this is because it is not finding the fortran libraries needed. These should come with cygwin, but I'm not sure how to point to them. Also it still reports not finding a fortran compiler, although g77 is installed in cygwin. Looking through the code I'm wondering if there is some confusion between use of os.name (which is 'nt') and sys.platform (which is 'win32') - I thought perhaps that's why its reporting not Fortran compilers supported for 'nt'. Not too sure though. Again any help greatfully received.
... lots of similar undefinied references ... C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0xe): undefined reference to `_s_wsfe' C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0x29): undefined reference to `_do_fio' C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0x44): undefined reference to `_do_fio' C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0x49): undefined reference to `_e_wsfe' C:\/libf77blas.a(xerbla.o):xerbla.f:(.text+0x5d): undefined reference to `_s_stop' collect2: ld returned 1 exit status error: Command "g++ -mno-cygwin -shared build\temp.win32-2.5\Release\numpy\linal g\lapack_litemodule.o -LC:\ -Lc:\Python25\libs -Lc:\Python25\PCBuild -llapack -l f77blas -lcblas -latlas -lpython25 -lmsvcr71 -o build\lib.win32-2.5\numpy\linalg \lapack_lite.pyd" failed with exit status 1
Thanks,
Robin
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- . __ . |-\ . . tim.hochberg@ieee.org

Hi, What version of MSVC are you using ? If you really want to have an optimized version, don't use mingw (gcc is not up to date). Matthieu 2007/7/13, numpy-discussion@robince.ftml.net < numpy-discussion@robince.ftml.net>:
Hi,
I am keen to evaluate numpy as an alternative to MATLAB for my PhD work and possible wider use within the department. To make a fairer comparison I wanted to build it with optimised ATLAS/LAPACK etc. hence building from source.
I am running into some problems though.
I am using Windows XP SP2, with latest Cygwin and I'm trying to follow the instructions on the wiki.
Firstly, is what I'm trying possible? On the Installing Scipy/Windows page it says MinGW gcc and g77 are best supported, but also says to build against binary Python download you need to use MSVC. From http://boodebr.org/main/python/build-windows-extensions it seems building extensions with gcc is fine (and I built PyCrypto successfully as a test). So can I do what I am trying to do (build numpy/scipy on windows using cygwin without MSVC installed) against the downloaded Python distribution? If not, I can't find any resources about building Python from source on Windows using Cygwin, so it seems like I would be completely stuck.
The next problem is that although I filled in the site.cfg file as documented (below), the setup.py script doesn't seem to pick it up and doesn't look in any of the specified directories. I can get around this by putting ATLAS/LAPACK libs in C:\, but obviously this isn't very satisfactory. Also is the entry for fftw correct? I couldn't find any information about this on the wiki.
------ [blas_opt] libraries = f77blas, cblas, atlas library_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\lib include_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\include # [lapack_opt] libraries = lapack, f77blas, cblas, atlas library_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\lib include_dirs = C:\cygwin\home\mqbxfri2\ATLAS\atlas_win32\include [fftw] libraries = fftw3-3 library_dirs = "C:\fftw" -----
Following this there seem to be some more problems with the setup.
"python setup.py build --compiler=mingw32" fails with: File
"C:\cygwin\home\mqbxfri2\numpy_trunk\numpy\distutils\fcompiler\__init__.py", line 731, in _find_existing_fcompiler c.customize(dist) AttributeError: 'NoneType' object has no attribute 'customize'
c is the result of the new_fcompiler function.
"python setup.py config_fc --help-fcompiler" failes with: File "c:\Python25\lib\distutils\msvccompiler.py", line 270, in initialize "version of the compiler, but it isn't installed." % self.__product) distutils.errors.DistutilsPlatformError: Python was built with Visual Studio version 7.1, and extensions need to be built with the same version of the compiler, but it isn't installed.
Although as I mentioned I can successfully build extensions with gcc.
Reading through the help I saw that there is a "none" fcompiler type, so using that I get a bit further:
"python setup.py build --compiler=mingw32 --fcompiler=none" run initially with build directory removed gives the same NoneType error, but then running it again the build appears to start. I then run into bug #220 http://projects.scipy.org/scipy/numpy/ticket/220:
numpy\core\src\multiarraymodule.c: In function `initmultiarray': numpy\core\src\multiarraymodule.c:7563: error: `NPY_ALLOW_THREADS' undeclared (first use in this function) numpy\core\src\multiarraymodule.c:7563: error: (Each undeclared identifier is reported only once numpy\core\src\multiarraymodule.c:7563: error: for each function it appears in.)
I would really appreciate any help to get this working. I understand building numpy doesn't require a fortran compiler, but scipy does. I am hoping to build scipy as well, so presumably the config system needs to recognise the cygwin g77 compiler for that to work? During the config I also see the following message: don't know how to compile Fortran code on platform 'nt' with 'gnu' compiler. Supported compilers are: absoft Does this mean it isn't possible to build scipy on Windows with Cygwin compilers?
If I am eventually successful I would be happy to update the wiki with some more detailed instructions based on my experiences.
Finally, I can't find any discussion of the relevant merits of ATLAS vs MKL, other than the different licensing. Is it expected that MKL performs better? Which is recommended?
Thanks very much,
Robin
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

I am using Windows XP SP2, with latest Cygwin and I'm trying to follow the instructions on the wiki.
Firstly, is what I'm trying possible? On the Installing Scipy/Windows page it says MinGW gcc and g77 are best supported, but also says to build against binary Python download you need to use MSVC.
With python 2.5, you can build extensions with MinGW. I haven't tried to build numpy though.
So can I do what I am trying to do (build numpy/scipy on windows using cygwin without MSVC installed) against the downloaded Python distribution?
CygWin is NOT the same as MinGW. They both are gcc, but IIUC, cygwin builds against the cygwin unix-like standard libs, and MinGW builds against the system libs -- so only MinGW can link with a Python build with MSVC. I can't help you with MinGW and either g77 or BLAS/LAPACK though. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Hi, Thanks for the responses. On Fri, 13 Jul 2007 15:37:52 -0700, "Christopher Barker" <Chris.Barker@noaa.gov> said:
CygWin is NOT the same as MinGW. They both are gcc, but IIUC, cygwin builds against the cygwin unix-like standard libs, and MinGW builds against the system libs -- so only MinGW can link with a Python build with MSVC.
Cygwin has mingw packages which provide the MinGW libs, so that it effectively provides a MinGW install that can be used outside of Cygwin. Since Cygwin is required to build lapack and atlas (for make etc.) it doesn't make much sense to install MinGW seperately on its own. Anyway, the compiler is definitely working with Python because I successfully built the PyCrypto extension as a test. On Fri, 13 Jul 2007 16:52:35 +0200, "Matthieu Brucher" <matthieu.brucher@gmail.com> said:
What version of MSVC are you using ?
If you really want to have an optimized version, don't use mingw (gcc is not up to date).
I'm not using MSVC - I was trying to build it with Cygwin/MinGW since it said on the Wiki that is what is best supported. On Fri, 13 Jul 2007 11:06:35 -0400, "Benjamin M. Schwartz" <bmschwar@fas.harvard.edu> said:
Far and away the easiest solution is to install Gentoo Linux. All you need to do under gentoo is to add the "lapack" useflag and then "emerge numpy". Gentoo will automatically install ATLAS/LAPACK by default, compiled according to your settings, and then compile NumPy to use that LAPACK.
I managed to successfully build everything on Linux without too much trouble. I guess what I'm trying to do on Windows isn't really possible at the moment - there seem to be some serious issues with distutils. As I mentioned, I can't get it to pick up any path settings from site.cfg, there is the bug with new_fcompiler returning None when a Compiler type is expected, it doesn't recognise the Cygwin/MinGW Fortran compiler, it appears to give the wrong linking flags for gcc from MinGW and also sometimes gives errors that MSVC is required, which is not true with Python 2.5. It seems most people building on Windows use MSVC, so I guess that is what I would try next, but for the time being I think I will try the coLinux suggestion. I was worried about matplotlib graphics over an X connection, but I guess there are other options like VNC etc. Perhaps the windows install section wiki could/should be updated to reflect the fact it's not currently possible to do this. Might save someone else some time! As for my unrelated question, I was still wondering if anyone has any information about the relative merits of MKL vs ATLAS etc. Thanks, Robin

numpy-discussion@robince.ftml.net wrote:
I am keen to evaluate numpy as an alternative to MATLAB for my PhD work and possible wider use within the department. To make a fairer comparison I wanted to build it with optimised ATLAS/LAPACK etc. hence building from source.
Far and away the easiest solution is to install Gentoo Linux. All you need to do under gentoo is to add the "lapack" useflag and then "emerge numpy". Gentoo will automatically install ATLAS/LAPACK by default, compiled according to your settings, and then compile NumPy to use that LAPACK. As others have noted, running linux in a virtual machine under Windows may still be faster and easier than trying to configure a working installation in Windows. This is especially true with coLinux, which has near-zero overhead. --Ben
participants (6)
-
Benjamin M. Schwartz
-
Christopher Barker
-
Matthieu Brucher
-
numpy-discussion@robince.ftml.net
-
Robin Ince
-
Timothy Hochberg