[Numpy-discussion] draft release guide

[Francesc Alted]

A Thursday 25 March 2010 02:00:36 David Cournapeau escrigué:
> Hosted compiler refers to the platform the compiler itself runs on (so
> here I mean a native 64 bits compiler, instead of a 32 bits compiler
> which targets 64 bits). It is nice that mingw-w64 gives a 64 bits
> hosted, that's recent.

Yeah, but unfortunately it is not enough for numpy to work.  After creating 
the binary package (thanks for the "build -c mingw32 bdist_wininst" trick), 
and installing nose, I get a big crash when running the test units.  When 
trying to use gdb to see the backtrace, I got:

C:\Users\francesc\Desktop\NumPy>gdb python
GNU gdb (GDB) 7.1.50.20100318-cvs
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-w64-mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from \Python26_64/python.exe...(no debugging symbols 
found)...done.
(gdb) r
Starting program: \Python26_64/python.exe
[New Thread 2880.0x410]
Python 2.6.5 (r265:79096, Mar 19 2010, 18:02:59) [MSC v.1500 64 bit (AMD64)] 
on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
C:\Python26_64\lib\site-packages\numpy\core\__init__.py:5: Warning: Numpy 
built
with MINGW-W64 on Windows 64 bits is experimental, and only available for
testing. You are advised not to use it for production.

CRASHES ARE TO BE EXPECTED - PLEASE REPORT THEM TO NUMPY DEVELOPERS
  import multiarray
>>> numpy.test()
Running unit tests for numpy
NumPy version 1.4.0
NumPy is installed in C:\Python26_64\lib\site-packages\numpy
Python version 2.6.5 (r265:79096, Mar 19 2010, 18:02:59) [MSC v.1500 64 bit 
(AMD
64)]
nose version 0.11.3
Forcing DISTUTILS_USE_SDK=1
.....warning: HEAP[python.exe]:
warning: Invalid address specified to RtlFreeHeap( 0000000002240000, 
0000000000C
25110 )


Program received signal SIGTRAP, Trace/breakpoint trap.
0x0000000076fa6061 in ntdll!DbgBreakPoint ()
   from C:\Windows\system32\ntdll.dll
(gdb) bt
#0  0x0000000076fa6061 in ntdll!DbgBreakPoint ()
   from C:\Windows\system32\ntdll.dll
#1  0x0000000076ffe17a in ntdll!EtwEventProviderEnabled ()
   from C:\Windows\system32\ntdll.dll
#2  0x00000000022af0d8 in ?? ()
#3  0x000000005104095c in ?? ()
#4  0x0000000000219a08 in ?? ()
#5  0x000000000e040001 in ?? ()
#6  0x0000000002240000 in ?? ()
#7  0x0000000076fe27a1 in ntdll!MD4Final () from C:\Windows\system32\ntdll.dll
#8  0x0000000076fb9630 in ntdll!LdrGetProcedureAddress ()
   from C:\Windows\system32\ntdll.dll
#9  0x0000000076fb9500 in ntdll!LdrGetProcedureAddress ()
   from C:\Windows\system32\ntdll.dll
#10 0x0000000002240000 in ?? ()
#11 0x0000000000c25110 in ?? ()
#12 0x0000000002240000 in ?? ()
#13 0x000000000623f197 in ?? ()
#14 0x0000000002240000 in ?? ()
#15 0x00000000770151a9 in ntdll!RtlTraceDatabaseCreate ()
   from C:\Windows\system32\ntdll.dll
#16 0x0000000000000000 in ?? ()
(gdb)

So, it is still exactly as you said: it seems like gdb cannot introspect MS 
debug symbols.  Unfortunately I don't think solving this would be easy at all 
:-/  Perhaps reporting this to mingw-w64 would help...

> > With it, and with some fixes in numpy sources (very few), I achieved to
> > pass the build phase.  I can provide the patch in case someone is
> > interested.
> 
> Building should work out of the box, actually - what are the issues ?

I'm attaching the patch.  Caveat emptor: the patch is *very* crude, and it is 
just meant to allow a clean compile with mingw-w64.  A more elaborated patch 
should be implemented in case we want it to go into numpy.

> But the main problem is not to build numpy (the 1.3.0 64 bits msi was
> built with mingw-w64 already) - it is to find out what causes the
> various crashes people encountered. Some were due to bugs in mingw which
> have been fixed since then.

Exactly.  OK, I think I'm done with this try for the moment.  It is a pity 
because mingw-w64 does an excellent job with my preliminary tests with Blosc 
compressor (it achieves 4x better performance than mingw-w32 and 2x better 
than MSVC 2008 32-bit).  But before going MSVC 64-bit route, I'd like to test 
Intel compiler 64-bit.  Anyone knows if it can cope with numpy?  A quick look 
at setup.py seems to say that only 32-bit Intel compiler is supported.

Thanks,

-- 
Francesc Alted
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mingw64.patch
Type: text/x-patch
Size: 2011 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100325/50db2436/attachment.bin>