Mailman 3 November 2012 - NumPy-Discussion

Compiling NumPy on Windows for Python 3.3 with MSVC 2010
by Peter Cock 20 Nov '12

20 Nov '12

Changing title to reflect the fact this thread is now about using the Microsoft compiler rather than mingw32 as in the old thread. On Sat, Nov 10, 2012 at 11:04 PM, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: > On Sat, Nov 10, 2012 at 5:47 PM, Ralf Gommers <ralf.gommers(a)gmail.com> wrote: >> >> >> >> On Tue, Nov 6, 2012 at 6:49 PM, Peter Cock <p.j.a.cock(a)googlemail.com> >> wrote: >>> >>> Dear all, >>> >>> Since the NumPy 1.7.0b2 release didn't include a Windows >>> (32 bit) installer for Python 3.3, I am considering compiling it >>> myself for local testing. What compiler is recommended? >> >> >> Either MSVC or MinGW 3.4.5. For the latter see >> https://github.com/certik/numpy-vendor > > Thanks Ralf, > > I was trying with MSVC 9.0 installed, but got this cryptic error: > > C:\Downloads\numpy-1.7.0b2 > C:\python33\python setup.py build > ... > error: Unable to find vcvarsall.bat > > After sprinkling distutils with debug statements, I found it was > looking for MSVC v10 (not numpy's fault but the error is most > unhelpful). > > Presumably "Microsoft Visual C++ 2010 Express Edition" is > the appropriate thing to download? > http://www.microsoft.com/visualstudio/eng/downloads#d-2010-express > I would have tried this earlier, but it required Windows XP SP3 and there was something amiss with the permissions on my work Windows machine preventing that update. Solved now, and this file now exists: C:\Program Files\Microsoft Visual Studio 10.0\VC\vcvarsall.bat I've tried my branch (focussed on mingw32 fixes), and numpy-1.7.0b2 but both fail at the same point. I did remove the old build directory first. C:\Downloads\numpy-1.7.0b2>c:\python33\python setup.py build Converting to Python3 via 2to3... <snip> Could not locate executable efc don't know how to compile Fortran code on platform 'nt' C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Inumpy\core\src\private -Inumpy\core\src -Inumpy\core -Inumpy\ core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\src\npysort -Inumpy\core\include -Ic:\python33\include -Ic:\python33\include /Tc_configtest.c /Fo_configtest.obj Found executable C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\cl.exe C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe /nologo /INCREMENTAL:NO _configtest.obj /OUT:_configtest.exe /MANIFESTFILE:_configtest.exe.manifest Found executable C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe mt.exe -nologo -manifest _configtest.exe.manifest -outputresource:_configtest.exe;1 Found executable C:\Program Files\Microsoft SDKs\Windows\v7.0A\bin\mt.exe _configtest.exe.manifest : general error c1010070: Failed to load and parse the manifest. The system cannot find the file specified. failure. removing: _configtest.c _configtest.obj Traceback (most recent call last): File "setup.py", line 214, in <module> setup_package() File "setup.py", line 207, in setup_package configuration=configuration ) File "C:\Downloads\numpy-1.7.0b2\build\py3k\numpy\distutils\core.py", line 186 , in setup return old_setup(**new_attr) File "c:\python33\lib\distutils\core.py", line 148, in setup dist.run_commands() File "c:\python33\lib\distutils\dist.py", line 917, in run_commands self.run_command(cmd) File "c:\python33\lib\distutils\dist.py", line 936, in run_command cmd_obj.run() File "C:\Downloads\numpy-1.7.0b2\build\py3k\numpy\distutils\command\build.py", line 37, in run old_build.run(self) File "c:\python33\lib\distutils\command\build.py", line 126, in run self.run_command(cmd_name) File "c:\python33\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "c:\python33\lib\distutils\dist.py", line 936, in run_command cmd_obj.run() File "C:\Downloads\numpy-1.7.0b2\build\py3k\numpy\distutils\command\build_src. py", line 152, in run self.build_sources() File "C:\Downloads\numpy-1.7.0b2\build\py3k\numpy\distutils\command\build_src. py", line 163, in build_sources self.build_library_sources(*libname_info) File "C:\Downloads\numpy-1.7.0b2\build\py3k\numpy\distutils\command\build_src. py", line 298, in build_library_sources sources = self.generate_sources(sources, (lib_name, build_info)) File "C:\Downloads\numpy-1.7.0b2\build\py3k\numpy\distutils\command\build_src. py", line 385, in generate_sources source = func(extension, build_dir) File "numpy\core\setup.py", line 648, in get_mathlib_info raise RuntimeError("Broken toolchain: cannot link a simple C program") RuntimeError: Broken toolchain: cannot link a simple C program It appears a similar issue was raised before: http://mail.scipy.org/pipermail/numpy-discussion/2012-June/062866.html Any tips? Peter

2 6

PRs for MaskedArray bugs
by Thomas Robitaille 20 Nov '12

20 Nov '12

I've recently opened a couple of pull requests that fix bugs with MaskedArray - these are pretty straightforward, so would it be possible to consider them in time for 1.7? https://github.com/numpy/numpy/pull/2703 https://github.com/numpy/numpy/pull/2733 Thanks! Tom

3 2

float32 to float64 casting
by Gökhan Sever 17 Nov '12

17 Nov '12

Hello, Could someone briefly explain why are these two operations are casting my float32 arrays to float64? I1 (np.arange(5, dtype='float32')).dtype O1 dtype('float32') I2 (100000*np.arange(5, dtype='float32')).dtype O2 dtype('float64') I3 (np.arange(5, dtype='float32')[0]).dtype O3 dtype('float32') I4 (1*np.arange(5, dtype='float32')[0]).dtype O4 dtype('float64') Thanks. -- Gökhan

5 10

Assignment function with a signature similar to take?
by Eric Moore 17 Nov '12

17 Nov '12

Is there a function that operates like 'take' but does assignment? Specifically that takes indices and an axis? As far as I can tell no such function exists. Is there any particular reason? One can fake such a thing by doing (code untested): s = len(a.shape)*[np.s_[:]] s[axis] = np.s_[1::2] a[s] = b.take(np.arange(1,b.shape[axis],2), axis) Or by using np.rollaxis: a = np.rollaxis(a, axis, len(a.shape)) a[..., 1::2] = b[..., 1::2] a = np.rollaxis(a, len(a.shape)-1, axis) But I don't really think that either of these are particularly clear, but probably prefer the rollaxis solution. Also, while I'm here, what about having take also be able to use a slice object in lieu of a collection of indices? -Eric

1 0

Segfault for Numpy 1.4.1 on 64-bit CentOS-5
by Tom Aldcroft 16 Nov '12

16 Nov '12

For library compatibility testing I'm trying to use numpy 1.4.1 with Python 2.7.3 on a 64-bit CentOS-5 platform. I installed a clean Python from source (basically "./configure --prefix=$prefix ; make install") and then installed numpy 1.4.1 with "python setup.py install". The crash message begins with [1]: *** glibc detected *** /home/aldcroft/vpy/py27_np141/bin/python: free(): invalid next size (fast): 0x000000001a9fcf30 *** ======= Backtrace: ========= /lib64/libc.so.6[0x31a90711df] /lib64/libc.so.6(cfree+0x4b)[0x31a907163b] /home/aldcroft/vpy/py27_np141/lib/python2.7/site-packages/numpy/core/multiarray.so[0x2aaab025ebbe] A problem which seems related is that fancy indexing is failing: >>> idx = np.array([1]) >>> idx.dtype dtype('int64') >>> np.arange(5)[idx] Traceback (most recent call last): File "<stdin>", line 1, in <module> IndexError: index 210453397505 out of bounds 0<=index<5 Does anyone have suggestions for compilation flags or workarounds when building numpy or Python that might fix this? Thanks, Tom [1] I don't know the exact code that triggered this crash, it's buried in some units tests. If it was useful I could dig it out.

1 0

Re: [Numpy-discussion] float32 to float64 casting
by Olivier Delalleau 16 Nov '12

16 Nov '12

2012/11/16 Olivier Delalleau <olivier.delalleau(a)gmail.com> > 2012/11/16 Charles R Harris <charlesr.harris(a)gmail.com> > >> >> >> On Thu, Nov 15, 2012 at 11:37 PM, Charles R Harris < >> charlesr.harris(a)gmail.com> wrote: >> >>> >>> >>> On Thu, Nov 15, 2012 at 8:24 PM, Gökhan Sever <gokhansever(a)gmail.com>wrote: >>> >>>> Hello, >>>> >>>> Could someone briefly explain why are these two operations are casting >>>> my float32 arrays to float64? >>>> >>>> I1 (np.arange(5, dtype='float32')).dtype >>>> O1 dtype('float32') >>>> >>>> I2 (100000*np.arange(5, dtype='float32')).dtype >>>> O2 dtype('float64') >>>> >>> >>> This one is depends on the size of the multiplier and is first present >>> in 1.6.0. I suspect it is a side effect of making the type conversion code >>> sensitive to magnitude. >>> >>> >>>> >>>> >>>> >>>> I3 (np.arange(5, dtype='float32')[0]).dtype >>>> O3 dtype('float32') >>>> >>>> I4 (1*np.arange(5, dtype='float32')[0]).dtype >>>> O4 dtype('float64') >>>> >>> >>> This one probably depends on the fact that the element is a scalar, but >>> doesn't look right. Scalars are promoted differently. Also holds in numpy >>> 1.5.0 so is of old provenance. >>> >>> >> This one has always bothered me: >> >> In [3]: (-1*arange(5, dtype=uint64)).dtype >> Out[3]: dtype('float64') >> > > My interpretation here is that since the possible results when multiplying > an int64 with an uint64 can be signed, and can go beyond the range of > int64, numpy prefers to cast everything to float64, which can represent > (even if approximately) a larger range of signed values. > Actually, thinking about it a bit more, I suspect the logic is not related to the result of the operation, but to the fact numpy needs to cast both arguments into a common dtype before doing the operation, and it has no integer dtype available that can hold both int64 and uint64 numbers, so it uses float64 instead. -=- Olivier

1 0

Numpydoc warnings for methods
by Jose Miguel Ibáñez 15 Nov '12

15 Nov '12

Hello, Now, I am having the same problem, and although I have tried the Pauili fix (see below) I still have the same problem when using numpydoc extension. Does anyone have more information or suggestions about it ? Thanks, Jose On Sun, Jul 17, 2011 at 7:15 PM, Tony Yu <tsyu80(a)gmail.com <http://mail.scipy.org/mailman/listinfo/numpy-discussion>> wrote: >* I'm building documentation using Sphinx, and it seems that numpydoc is*>* raising*>* a lot of warnings. Specifically, the warnings look like "failed to import*>* <method_name>", "toctree*>* references unknown document u'<method_name>'", "toctree contains reference*>* to nonexisting document '<method_name>'---for each method defined. The*>* example below reproduces the issue on my system (Sphinx 1.0.7, numpy HEAD).*>* These warnings appear in my build of the numpy docs, as well.*>**>* Removing numpydoc from the list of Sphinx extensions gets rid of these*>* warnings*>* (but, of course, adds new warnings if headings for 'Parameters', 'Returns',*>* etc. are present).*>**>* Am I doing something wrong here?*>**>* You're not, it's a Sphinx bug that Pauli already has a fix for. See*http://projects.scipy.org/numpy/ticket/1772 Ralf

1 0

numpy.power vs pylab.power
by Will Furnass 15 Nov '12

15 Nov '12

On my machine these are rather confusingly different functions, with the latter corresponding to numpy.random.power. I appreciate that pylab imports everything from both the numpy and numpy.random modules but wouldn't it make sense if pylab.power were the frequently used power function rather than a means for sampling from the power distribution? Regards, Will Furnass

2 1

Should abs([nan]) be supported?
by Ondřej Čertík 14 Nov '12

14 Nov '12

Hi, When running the test suite, there are problems of this kind: https://github.com/numpy/numpy/issues/394 which then causes for example the Debian buildbots tests to fail (https://github.com/numpy/numpy/issues/406). The problem is really simple: >>> from numpy import array, abs, nan >>> a = array([1, nan, 3]) >>> a array([ 1., nan, 3.]) >>> abs(a) __main__:1: RuntimeWarning: invalid value encountered in absolute array([ 1., nan, 3.]) See the issue #394 for detailed explanation why "nan" is being passed to abs(). Now the question is, what should the right fix be? 1) Should the runtime warning be disabled? 2) Should the tests be reworked, so that "nan" is not tested in allclose()? 3) Should abs() be fixed to not emit the warning? 4) Should the test suite be somehow fixed not to fail if there are runtime warnings? Let me know which direction we should go. Thanks, Ondrej

3 5

1.7.0 release
by Travis Oliphant 14 Nov '12

14 Nov '12

Hey all, Ondrej has been tied up finishing his PhD for the past several weeks. He is defending his work shortly and should be available to continue to help with the 1.7.0 release around the first of December. He and I have been in contact during this process, and I've been helping where I can. Fortunately, other NumPy developers have been active closing tickets and reviewing pull requests which has helped the process substantially. The release has taken us longer than we expected, but I'm really glad that we've received the bug-reports and issues that we have seen because it will help the 1.7.0 release be a more stable series. Also, the merging of the Trac issues with Git has exposed over-looked problems as well and will hopefully encourage more Git-focused participation by users. We are targeting getting the final release of 1.7.0 out by mid December (based on Ondrej's availability). But, I would like to find out which issues are seen as blockers by people on this list. I think most of the issues that I had as blockers have been resolved. If there are no more remaining blockers, then we may be able to accelerate the final release of 1.7.0 to just after Thanksgiving. Best regards, -Travis

10 14