Hi all,
At http://github.com/rgommers/NumPy-release-guide you can find a summary of how to set up your system to build numpy binaries on OS X. I still have to add info on scipy (that's turning out to be fairly painful) but for numpy it is pretty complete.
Any feedback is appreciated!
Cheers, Ralf
Ralf Gommers wrote:
At http://github.com/rgommers/NumPy-release-guide you can find a summary of how to set up your system to build numpy binaries on OS X. I still have to add info on scipy (that's turning out to be fairly painful) but for numpy it is pretty complete.
Any feedback is appreciated!
We really don't want to supply binaries for python 2.5 anymore?
(maybe this will get me to finally switch!
-Chris
Maybe I missed the discussion, but is there a reason why we don't want to support Python 2.5 via providing binaries?
I'm working on a detailed write up of how to create windows binaries on Windows 7. I hope to finish in the next day or so, however, my brother-in-law made an unexpected visit, so I don't have as much time as I thought I did. With that said, I hope to be able to share it by this weekend.
Patrick
On Tue, Mar 23, 2010 at 12:40 PM, Christopher Barker Chris.Barker@noaa.govwrote:
Ralf Gommers wrote:
At http://github.com/rgommers/NumPy-release-guide you can find a summary of how to set up your system to build numpy binaries on OS X. I still have to add info on scipy (that's turning out to be fairly painful) but for numpy it is pretty complete.
Any feedback is appreciated!
We really don't want to supply binaries for python 2.5 anymore?
(maybe this will get me to finally switch!
-Chris
-- Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Mar 24, 2010 at 1:40 AM, Christopher Barker Chris.Barker@noaa.govwrote:
Ralf Gommers wrote:
At http://github.com/rgommers/NumPy-release-guide you can find a summary of how to set up your system to build numpy binaries on OS X. I still have to add info on scipy (that's turning out to be fairly painful) but for numpy it is pretty complete.
Any feedback is appreciated!
We really don't want to supply binaries for python 2.5 anymore?
(maybe this will get me to finally switch!
That's fast, I was hoping to get away with that one!
I discussed a bit with David, and he suggested to not make 2.5 binaries for scipy 0.7.2 and then see if there was still any demand. It's a lot of work with perhaps not so much return. Of course a maintenance release of scipy with no new features may be a different story from a new release of numpy/scipy. So by no means is this decided or even publicly discussed. I do want to point out that we have not provided 2.4 binaries for a while without much demand.
So now the question: who still wants and uses 2.5 binaries?
Cheers, Ralf
A Wednesday 24 March 2010 01:49:50 Ralf Gommers escrigué:
On Wed, Mar 24, 2010 at 1:40 AM, Christopher Barker
Chris.Barker@noaa.govwrote:
Ralf Gommers wrote:
At http://github.com/rgommers/NumPy-release-guide you can find a summary of how to set up your system to build numpy binaries on OS X. I still have to add info on scipy (that's turning out to be fairly painful) but for numpy it is pretty complete.
Any feedback is appreciated!
We really don't want to supply binaries for python 2.5 anymore?
(maybe this will get me to finally switch!
That's fast, I was hoping to get away with that one!
I discussed a bit with David, and he suggested to not make 2.5 binaries for scipy 0.7.2 and then see if there was still any demand. It's a lot of work with perhaps not so much return. Of course a maintenance release of scipy with no new features may be a different story from a new release of numpy/scipy. So by no means is this decided or even publicly discussed. I do want to point out that we have not provided 2.4 binaries for a while without much demand.
So now the question: who still wants and uses 2.5 binaries?
I do. I think at least 2 binary versions is a recommendable practice for allowing people to change versions more comfortably.
Also, I have read the draft and I cannot see references to 64-bit binary packages. With the advent of Windows 7 and Mac OSX Snow Leopard, 64-bit are way more spread than before, so they would be a great thing to deliver, IMO.
Thanks,
On Wed, Mar 24, 2010 at 5:50 PM, Francesc Alted faltet@pytables.org wrote:
A Wednesday 24 March 2010 01:49:50 Ralf Gommers escrigué:
On Wed, Mar 24, 2010 at 1:40 AM, Christopher Barker
Chris.Barker@noaa.govwrote:
Ralf Gommers wrote:
At http://github.com/rgommers/NumPy-release-guide you can find a summary of how to set up your system to build numpy binaries on OS X.
I
still have to add info on scipy (that's turning out to be fairly painful) but for numpy it is pretty complete.
Any feedback is appreciated!
We really don't want to supply binaries for python 2.5 anymore?
(maybe this will get me to finally switch!
That's fast, I was hoping to get away with that one!
I discussed a bit with David, and he suggested to not make 2.5 binaries
for
scipy 0.7.2 and then see if there was still any demand. It's a lot of
work
with perhaps not so much return. Of course a maintenance release of scipy with no new features may be a different story from a new release of numpy/scipy. So by no means is this decided or even publicly discussed. I do want to point out that we have not provided 2.4 binaries for a while without much demand.
So now the question: who still wants and uses 2.5 binaries?
I do. I think at least 2 binary versions is a recommendable practice for allowing people to change versions more comfortably.
Okay thanks, that's two people already so I'll try.
Also, I have read the draft and I cannot see references to 64-bit binary packages. With the advent of Windows 7 and Mac OSX Snow Leopard, 64-bit are way more spread than before, so they would be a great thing to deliver, IMO.
The reason is that for OS X the binaries from python.org are 32-bit only so far. Apple Python is 64-bit, but seems to have its own issues (way behind with bug fix releases, maybe there's more). David told me to always only target the python.org version, which makes sense to me.
For Windows 7, I agree it's worth looking into 64-bits binaries, but can't tell you right now how easy that would be.
Cheers, Ralf
On Wed, Mar 24, 2010 at 6:50 PM, Francesc Alted faltet@pytables.org wrote:
Also, I have read the draft and I cannot see references to 64-bit binary packages. With the advent of Windows 7 and Mac OSX Snow Leopard, 64-bit are way more spread than before, so they would be a great thing to deliver, IMO.
For Mac OS X, we should wait for official 64 bits binaries from python.org - EPD offers 64 bits binaries if necessary. On windows, the situation is more complicated, because there is no way to build numpy and scipy correctly with free compilers. For various reasons, I think we should avoid distributing binaries based on non-free compilers (not available to everyone, problem of compatibility and redistribution of non-free code, especially for fortran) - and for people who don't mind/care, there are unofficial win64 binaries out there (we may put a link somewhere). Unfortunately, I don't think I will have much time to spend on making scipy and gfortran work together on win64 in the near future,
cheers,
David
A Wednesday 24 March 2010 12:00:36 David Cournapeau escrigué:
On Wed, Mar 24, 2010 at 6:50 PM, Francesc Alted faltet@pytables.org wrote:
Also, I have read the draft and I cannot see references to 64-bit binary packages. With the advent of Windows 7 and Mac OSX Snow Leopard, 64-bit are way more spread than before, so they would be a great thing to deliver, IMO.
For Mac OS X, we should wait for official 64 bits binaries from python.org - EPD offers 64 bits binaries if necessary. On windows, the situation is more complicated, because there is no way to build numpy and scipy correctly with free compilers. For various reasons, I think we should avoid distributing binaries based on non-free compilers (not available to everyone, problem of compatibility and redistribution of non-free code, especially for fortran) - and for people who don't mind/care, there are unofficial win64 binaries out there (we may put a link somewhere). Unfortunately, I don't think I will have much time to spend on making scipy and gfortran work together on win64 in the near future,
Ok. I've been having a try at mingw-w64 project:
http://mingw-w64.sourceforge.net/
with no success so far with build numpy:
$ python setup.py build --compiler=mingw32 [...] Found executable C:\mingw-w64_x86_64-mingw\mingw64\bin\gcc.exe g++ -mno-cygwin _configtest.o -lmsvcr90 -o _configtest.exe Found executable C:\mingw-w64_x86_64-mingw\mingw64\bin\g++.exe c:/mingw-w64_x86_64-mingw/mingw64/bin/../lib/gcc/x86_64-w64- mingw32/4.4.4/../../ ../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lmsvcr90 collect2: ld returned 1 exit status failure. [...]
However, I can compile it with plain mingw tool chain (obviously, only can get w32 binaries). Any hint on how to proceed with mingw-w64?
Thanks,
On Wed, Mar 24, 2010 at 11:25 PM, Francesc Alted faltet@pytables.org wrote:
Ok. I've been having a try at mingw-w64 project:
http://mingw-w64.sourceforge.net/
with no success so far with build numpy:
$ python setup.py build --compiler=mingw32
Oh, it is not that easy :)
First, for some reason, the mingw-w64 project does not provide 64 hosted compilers, and since pushing for mingw cross compilation support in distutils would redefine the meaning of insanity, I build my gcc. Since building gcc on windows is not a fun ride either, you have to build it on unix (I have the scripts on my github account: github.com/cournape/).
Then, you can hope to build numpy with mingw 64 on windows 64. But then, it randomly crashes, and last time I looked at it (~ 8 months ago), there was no gdb support and the mingw debug symbols cannot be understood by MS tools, so I was stuck at trying to understand what went wrong without a meaningful backtrace.
I also looked into making gfortran works with MS compiler, but this means porting libgfortran to something MS compilers can understand. This is not as crazy as it sounds, because we don't use many fortran runtime functions in scipy.
Of course, atlas on windows 64 is nowhere near buildable.
cheers,
David
A Wednesday 24 March 2010 15:38:58 David Cournapeau escrigué:
Oh, it is not that easy :)
First, for some reason, the mingw-w64 project does not provide 64 hosted compilers, and since pushing for mingw cross compilation support in distutils would redefine the meaning of insanity, I build my gcc. Since building gcc on windows is not a fun ride either, you have to build it on unix (I have the scripts on my github account: github.com/cournape/).
Mmh, not sure about what you mean by hosted compiler, but there is certainly a native compiler package for Win64:
mingw-w64-bin_x86_64-mingw_20100322_sezero.zip
It comes with gcc, g++ and gfortran 4.4.4. gdb support is also there. So it seems like a pretty complete toolset for windows amd64.
With it, and with some fixes in numpy sources (very few), I achieved to pass the build phase. I can provide the patch in case someone is interested. The generated extensions are:
24/03/2010 19:34 1.492.313 multiarray.pyd 24/03/2010 19:34 124.866 multiarray_tests.pyd 24/03/2010 19:34 453.377 scalarmath.pyd 24/03/2010 19:34 1.079.827 umath.pyd 24/03/2010 19:34 121.651 umath_tests.pyd 24/03/2010 19:34 304.014 _sort.pyd
which looks good to my eyes.
Now, when I try to generate the installable package I'm in trouble again:
$ python setup.py bdist [...] File "C: \Users\francesc\Desktop\NumPy\1.4.x\numpy\distutils\command\config.py" , line 56, in _check_compiler self.compiler.initialize() File "C:\Python26_64\lib\distutils\msvc9compiler.py", line 359, in initialize vc_env = query_vcvarsall(VERSION, plat_spec) File "C:\Python26_64\lib\distutils\msvc9compiler.py", line 275, in query_vcvar sall raise ValueError(str(list(result.keys()))) ValueError: [u'path'] [...]
So, it looks like either numpy or python cannot determine that the used compiler is mingw instead of msvc9. However, when I try to specify mingw explicitly, I get the next error:
$ python setup.py bdist --compiler=mingw32 Running from numpy source directory. Forcing DISTUTILS_USE_SDK=1 usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help
error: option --compiler not recognized
Someone could tell me why distutils can be told to use mingw32 compiler for the build stage but not for bdist? What is more, why the need for a compiler for bdist if numpy is already built? I feel that I'm almost there, but some piece still resists...
On Wed, Mar 24, 2010 at 3:31 PM, Francesc Alted faltet@pytables.org wrote:
A Wednesday 24 March 2010 15:38:58 David Cournapeau escrigué:
Oh, it is not that easy :)
First, for some reason, the mingw-w64 project does not provide 64 hosted compilers, and since pushing for mingw cross compilation support in distutils would redefine the meaning of insanity, I build my gcc. Since building gcc on windows is not a fun ride either, you have to build it on unix (I have the scripts on my github account: github.com/cournape/).
Mmh, not sure about what you mean by hosted compiler, but there is certainly a native compiler package for Win64:
mingw-w64-bin_x86_64-mingw_20100322_sezero.zip
It comes with gcc, g++ and gfortran 4.4.4. gdb support is also there. So it seems like a pretty complete toolset for windows amd64.
With it, and with some fixes in numpy sources (very few), I achieved to pass the build phase. I can provide the patch in case someone is interested. The generated extensions are:
24/03/2010 19:34 1.492.313 multiarray.pyd 24/03/2010 19:34 124.866 multiarray_tests.pyd 24/03/2010 19:34 453.377 scalarmath.pyd 24/03/2010 19:34 1.079.827 umath.pyd 24/03/2010 19:34 121.651 umath_tests.pyd 24/03/2010 19:34 304.014 _sort.pyd
which looks good to my eyes.
Now, when I try to generate the installable package I'm in trouble again:
$ python setup.py bdist [...] File "C: \Users\francesc\Desktop\NumPy\1.4.x\numpy\distutils\command\config.py" , line 56, in _check_compiler self.compiler.initialize() File "C:\Python26_64\lib\distutils\msvc9compiler.py", line 359, in initialize vc_env = query_vcvarsall(VERSION, plat_spec) File "C:\Python26_64\lib\distutils\msvc9compiler.py", line 275, in query_vcvar sall raise ValueError(str(list(result.keys()))) ValueError: [u'path'] [...]
So, it looks like either numpy or python cannot determine that the used compiler is mingw instead of msvc9. However, when I try to specify mingw explicitly, I get the next error:
$ python setup.py bdist --compiler=mingw32 Running from numpy source directory. Forcing DISTUTILS_USE_SDK=1 usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help
error: option --compiler not recognized
For some similar problems, which I don't remember exactly, I needed to create the dist in the same command as the build, e.g.
python setup.py build --compiler=mingw32 bdist
I don't know if this works in your case, I have the compiler specified in distutils.cfg
Josef
Someone could tell me why distutils can be told to use mingw32 compiler for the build stage but not for bdist? What is more, why the need for a compiler for bdist if numpy is already built? I feel that I'm almost there, but some piece still resists...
-- Francesc Alted _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
What happens if you try to build a windows installer "python setup.py bdist_wininst"
Also, have you attempted to specify the compiler in "$PYTHON_ROOT\Lib\distutils\distutils.cfg" ?
I've got a script I use to manually change the "build" section of this file between
[build] msvc
and
[build] mingw32
to switch between default compilers when I've had issues with setup.py.
On Wed, Mar 24, 2010 at 2:31 PM, Francesc Alted faltet@pytables.org wrote:
A Wednesday 24 March 2010 15:38:58 David Cournapeau escrigué:
Oh, it is not that easy :)
First, for some reason, the mingw-w64 project does not provide 64 hosted compilers, and since pushing for mingw cross compilation support in distutils would redefine the meaning of insanity, I build my gcc. Since building gcc on windows is not a fun ride either, you have to build it on unix (I have the scripts on my github account: github.com/cournape/).
Mmh, not sure about what you mean by hosted compiler, but there is certainly a native compiler package for Win64:
mingw-w64-bin_x86_64-mingw_20100322_sezero.zip
It comes with gcc, g++ and gfortran 4.4.4. gdb support is also there. So it seems like a pretty complete toolset for windows amd64.
With it, and with some fixes in numpy sources (very few), I achieved to pass the build phase. I can provide the patch in case someone is interested. The generated extensions are:
24/03/2010 19:34 1.492.313 multiarray.pyd 24/03/2010 19:34 124.866 multiarray_tests.pyd 24/03/2010 19:34 453.377 scalarmath.pyd 24/03/2010 19:34 1.079.827 umath.pyd 24/03/2010 19:34 121.651 umath_tests.pyd 24/03/2010 19:34 304.014 _sort.pyd
which looks good to my eyes.
Now, when I try to generate the installable package I'm in trouble again:
$ python setup.py bdist [...] File "C: \Users\francesc\Desktop\NumPy\1.4.x\numpy\distutils\command\config.py" , line 56, in _check_compiler self.compiler.initialize() File "C:\Python26_64\lib\distutils\msvc9compiler.py", line 359, in initialize vc_env = query_vcvarsall(VERSION, plat_spec) File "C:\Python26_64\lib\distutils\msvc9compiler.py", line 275, in query_vcvar sall raise ValueError(str(list(result.keys()))) ValueError: [u'path'] [...]
So, it looks like either numpy or python cannot determine that the used compiler is mingw instead of msvc9. However, when I try to specify mingw explicitly, I get the next error:
$ python setup.py bdist --compiler=mingw32 Running from numpy source directory. Forcing DISTUTILS_USE_SDK=1 usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help
error: option --compiler not recognized
Someone could tell me why distutils can be told to use mingw32 compiler for the build stage but not for bdist? What is more, why the need for a compiler for bdist if numpy is already built? I feel that I'm almost there, but some piece still resists...
-- Francesc Alted _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Francesc Alted wrote:
A Wednesday 24 March 2010 15:38:58 David Cournapeau escrigué:
Oh, it is not that easy :)
First, for some reason, the mingw-w64 project does not provide 64 hosted compilers, and since pushing for mingw cross compilation support in distutils would redefine the meaning of insanity, I build my gcc. Since building gcc on windows is not a fun ride either, you have to build it on unix (I have the scripts on my github account: github.com/cournape/).
Mmh, not sure about what you mean by hosted compiler
Hosted compiler refers to the platform the compiler itself runs on (so here I mean a native 64 bits compiler, instead of a 32 bits compiler which targets 64 bits). It is nice that mingw-w64 gives a 64 bits hosted, that's recent.
It comes with gcc, g++ and gfortran 4.4.4. gdb support is also there. So it seems like a pretty complete toolset for windows amd64.
The problem with gdb (but I would need to see if that's still true) is that it did not give useful traceback for code compiled with MS compiler , the stack was always corrupted somewhere for the crashes I encountered.
With it, and with some fixes in numpy sources (very few), I achieved to pass the build phase. I can provide the patch in case someone is interested.
Building should work out of the box, actually - what are the issues ?
which looks good to my eyes.
Now, when I try to generate the installable package I'm in trouble again:
$ python setup.py bdist [...] File "C: \Users\francesc\Desktop\NumPy\1.4.x\numpy\distutils\command\config.py" , line 56, in _check_compiler self.compiler.initialize() File "C:\Python26_64\lib\distutils\msvc9compiler.py", line 359, in initialize vc_env = query_vcvarsall(VERSION, plat_spec) File "C:\Python26_64\lib\distutils\msvc9compiler.py", line 275, in query_vcvar sall raise ValueError(str(list(result.keys()))) ValueError: [u'path'] [...]
So, it looks like either numpy or python cannot determine that the used compiler is mingw instead of msvc9.
Yes, you have to set it explicitly, this is no different than on 32 bits:
python setup.py build -c mingw32 bdist_wininst.
But the main problem is not to build numpy (the 1.3.0 64 bits msi was built with mingw-w64 already) - it is to find out what causes the various crashes people encountered. Some were due to bugs in mingw which have been fixed since then.
cheers,
David
A Thursday 25 March 2010 02:00:36 David Cournapeau escrigué:
Hosted compiler refers to the platform the compiler itself runs on (so here I mean a native 64 bits compiler, instead of a 32 bits compiler which targets 64 bits). It is nice that mingw-w64 gives a 64 bits hosted, that's recent.
Yeah, but unfortunately it is not enough for numpy to work. After creating the binary package (thanks for the "build -c mingw32 bdist_wininst" trick), and installing nose, I get a big crash when running the test units. When trying to use gdb to see the backtrace, I got:
C:\Users\francesc\Desktop\NumPy>gdb python GNU gdb (GDB) 7.1.50.20100318-cvs Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-w64-mingw32". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from \Python26_64/python.exe...(no debugging symbols found)...done. (gdb) r Starting program: \Python26_64/python.exe [New Thread 2880.0x410] Python 2.6.5 (r265:79096, Mar 19 2010, 18:02:59) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import numpy
C:\Python26_64\lib\site-packages\numpy\core__init__.py:5: Warning: Numpy built with MINGW-W64 on Windows 64 bits is experimental, and only available for testing. You are advised not to use it for production.
CRASHES ARE TO BE EXPECTED - PLEASE REPORT THEM TO NUMPY DEVELOPERS import multiarray
numpy.test()
Running unit tests for numpy NumPy version 1.4.0 NumPy is installed in C:\Python26_64\lib\site-packages\numpy Python version 2.6.5 (r265:79096, Mar 19 2010, 18:02:59) [MSC v.1500 64 bit (AMD 64)] nose version 0.11.3 Forcing DISTUTILS_USE_SDK=1 .....warning: HEAP[python.exe]: warning: Invalid address specified to RtlFreeHeap( 0000000002240000, 0000000000C 25110 )
Program received signal SIGTRAP, Trace/breakpoint trap. 0x0000000076fa6061 in ntdll!DbgBreakPoint () from C:\Windows\system32\ntdll.dll (gdb) bt #0 0x0000000076fa6061 in ntdll!DbgBreakPoint () from C:\Windows\system32\ntdll.dll #1 0x0000000076ffe17a in ntdll!EtwEventProviderEnabled () from C:\Windows\system32\ntdll.dll #2 0x00000000022af0d8 in ?? () #3 0x000000005104095c in ?? () #4 0x0000000000219a08 in ?? () #5 0x000000000e040001 in ?? () #6 0x0000000002240000 in ?? () #7 0x0000000076fe27a1 in ntdll!MD4Final () from C:\Windows\system32\ntdll.dll #8 0x0000000076fb9630 in ntdll!LdrGetProcedureAddress () from C:\Windows\system32\ntdll.dll #9 0x0000000076fb9500 in ntdll!LdrGetProcedureAddress () from C:\Windows\system32\ntdll.dll #10 0x0000000002240000 in ?? () #11 0x0000000000c25110 in ?? () #12 0x0000000002240000 in ?? () #13 0x000000000623f197 in ?? () #14 0x0000000002240000 in ?? () #15 0x00000000770151a9 in ntdll!RtlTraceDatabaseCreate () from C:\Windows\system32\ntdll.dll #16 0x0000000000000000 in ?? () (gdb)
So, it is still exactly as you said: it seems like gdb cannot introspect MS debug symbols. Unfortunately I don't think solving this would be easy at all :-/ Perhaps reporting this to mingw-w64 would help...
With it, and with some fixes in numpy sources (very few), I achieved to pass the build phase. I can provide the patch in case someone is interested.
Building should work out of the box, actually - what are the issues ?
I'm attaching the patch. Caveat emptor: the patch is *very* crude, and it is just meant to allow a clean compile with mingw-w64. A more elaborated patch should be implemented in case we want it to go into numpy.
But the main problem is not to build numpy (the 1.3.0 64 bits msi was built with mingw-w64 already) - it is to find out what causes the various crashes people encountered. Some were due to bugs in mingw which have been fixed since then.
Exactly. OK, I think I'm done with this try for the moment. It is a pity because mingw-w64 does an excellent job with my preliminary tests with Blosc compressor (it achieves 4x better performance than mingw-w32 and 2x better than MSVC 2008 32-bit). But before going MSVC 64-bit route, I'd like to test Intel compiler 64-bit. Anyone knows if it can cope with numpy? A quick look at setup.py seems to say that only 32-bit Intel compiler is supported.
Thanks,
Francesc Alted wrote:
C:\Users\francesc\Desktop\NumPy>gdb python GNU gdb (GDB) 7.1.50.20100318-cvs Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-w64-mingw32". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from \Python26_64/python.exe...(no debugging symbols found)...done. (gdb) r Starting program: \Python26_64/python.exe [New Thread 2880.0x410] Python 2.6.5 (r265:79096, Mar 19 2010, 18:02:59) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import numpy
C:\Python26_64\lib\site-packages\numpy\core__init__.py:5: Warning: Numpy built with MINGW-W64 on Windows 64 bits is experimental, and only available for testing. You are advised not to use it for production.
CRASHES ARE TO BE EXPECTED - PLEASE REPORT THEM TO NUMPY DEVELOPERS import multiarray
numpy.test()
Running unit tests for numpy NumPy version 1.4.0 NumPy is installed in C:\Python26_64\lib\site-packages\numpy Python version 2.6.5 (r265:79096, Mar 19 2010, 18:02:59) [MSC v.1500 64 bit (AMD 64)] nose version 0.11.3 Forcing DISTUTILS_USE_SDK=1 .....warning: HEAP[python.exe]: warning: Invalid address specified to RtlFreeHeap( 0000000002240000, 0000000000C 25110 )
Program received signal SIGTRAP, Trace/breakpoint trap. 0x0000000076fa6061 in ntdll!DbgBreakPoint () from C:\Windows\system32\ntdll.dll (gdb) bt #0 0x0000000076fa6061 in ntdll!DbgBreakPoint () from C:\Windows\system32\ntdll.dll #1 0x0000000076ffe17a in ntdll!EtwEventProviderEnabled () from C:\Windows\system32\ntdll.dll #2 0x00000000022af0d8 in ?? () #3 0x000000005104095c in ?? () #4 0x0000000000219a08 in ?? () #5 0x000000000e040001 in ?? () #6 0x0000000002240000 in ?? () #7 0x0000000076fe27a1 in ntdll!MD4Final () from C:\Windows\system32\ntdll.dll #8 0x0000000076fb9630 in ntdll!LdrGetProcedureAddress () from C:\Windows\system32\ntdll.dll #9 0x0000000076fb9500 in ntdll!LdrGetProcedureAddress () from C:\Windows\system32\ntdll.dll #10 0x0000000002240000 in ?? () #11 0x0000000000c25110 in ?? () #12 0x0000000002240000 in ?? () #13 0x000000000623f197 in ?? () #14 0x0000000002240000 in ?? () #15 0x00000000770151a9 in ntdll!RtlTraceDatabaseCreate () from C:\Windows\system32\ntdll.dll #16 0x0000000000000000 in ?? () (gdb)
Believe it or not, but this is already much better than what I had last time I looked at it (the stack was corrupted after two items, and gdb often crashed). I had to build custom mingw runtimes to get there last year :)
So, it is still exactly as you said: it seems like gdb cannot introspect MS debug symbols.
And for completeness, the MS tools of course cannot read the mingw debugging symbols, but the contrary would have been surprising. It does not help that debugging things inside the MS C runtime is a nightmare (because you have to rebuild everything, including python). I wonder if MS makes the sources of their runtime available under some license to look more into it.
I also thought about using valgrind, but then there are some issues running wine in 64 bits mode under valgrind (maybe solved since then: https://bugs.kde.org/show_bug.cgi?id=197259).
Exactly. OK, I think I'm done with this try for the moment. It is a pity because mingw-w64 does an excellent job with my preliminary tests with Blosc compressor (it achieves 4x better performance than mingw-w32 and 2x better than MSVC 2008 32-bit). But before going MSVC 64-bit route, I'd like to test Intel compiler 64-bit. Anyone knows if it can cope with numpy?
What works well is MS compiler + Intel Fortran compiler (at least with numscons). Surprisingly, when I tried using the Intel C compiler + Intel Fortran compiler, I also had a lot of issues, not unlike the ones with mingw. What I am afraid is that the C runtimes issues are unsolvable on win64 because of python, and that we have to use the MS compiler (for C and C++).
cheers,
David
A Thursday 25 March 2010 10:53:34 David Cournapeau escrigué:
Believe it or not, but this is already much better than what I had last time I looked at it (the stack was corrupted after two items, and gdb often crashed). I had to build custom mingw runtimes to get there last year :)
Well, I've reported the problem just in case:
https://sourceforge.net/projects/mingw-w64/forums/forum/723797/topic/3638920
What works well is MS compiler + Intel Fortran compiler (at least with numscons). Surprisingly, when I tried using the Intel C compiler + Intel Fortran compiler, I also had a lot of issues, not unlike the ones with mingw. What I am afraid is that the C runtimes issues are unsolvable on win64 because of python, and that we have to use the MS compiler (for C and C++).
Ok. So it seems the MS compiler venue for 64-bit is unavoidable (at this moment, at least). One question though: is a fortran compiler really necessary for compiling just numpy? If so, why?
On Thu, Mar 25, 2010 at 9:33 PM, Francesc Alted faltet@pytables.org wrote:
Ok. So it seems the MS compiler venue for 64-bit is unavoidable (at this moment, at least). One question though: is a fortran compiler really necessary for compiling just numpy?
No, but I personally have little interest in numpy without scipy. You can build numpy with MS compilers on 64 bits since 1.3.0, this is entirely supported.
David
Francesc Alted wrote:
Also, I have read the draft and I cannot see references to 64-bit binary packages. With the advent of Windows 7 and Mac OSX Snow Leopard, 64-bit are way more spread than before, so they would be a great thing to deliver, IMO.
True, however the situation is a bit ugly with OS-X -- python.org does not distribute 64 bit binaries at all, and Apple's Python is a bit weird in that regard. Also some key libraries (anything build on Carbon , such as wxPython) don't work on 64 bit.
That being said, a 32+64 bit build for the Apple supplied Python for OS-X 10.6 might be nice, though a bit of a niche market.
-Chris
participants (8)
-
Chris Barker -
Christopher Barker
-
David Cournapeau -
David Cournapeau -
Francesc Alted -
josef.pktd＠gmail.com -
Patrick Marsh -
Ralf Gommers