Mailman 3 February 2016 - NumPy-Discussion

Re: [Numpy-discussion] Fwd: Windows wheels for testing
by R Schumacher 13 Feb '16

13 Feb '16

Have you all conferred with C Gohlke on his Windows build bot? I've never seen a description of his recipes. The MKL linking aside, his binaries always seem to work flawlessly. - Ray At 11:16 PM 2/12/2016, you wrote: >AFAIK the vcvarsall.bat error occurs when your >MSVC directories aren't properly linked in your >system registry, so Python cannot find the >file.Â This is not a numpy-specific issue, so I >certainly would agree that that failure is not blocking. > >Other than that, this build contains the >mingw32.lib bug that I fixedÂ ><https://github.com/numpy/numpy/pull/7119>here, >but other than that, everything else passes on >relevant Python versions for 32-bit! > >On Sat, Feb 13, 2016 at 4:23 AM, Matthew Brett ><<mailto:matthew.brett@gmail.com>matthew.brett(a)gmail.com> wrote: >On Fri, Feb 12, 2016 at 8:18 PM, R Schumacher ><<mailto:rays@blue-cove.com>rays(a)blue-cove.com> wrote: > > At 03:45 PM 2/12/2016, you wrote: > >> > >> PS C:\tmp> c:\Python35\python -m venv np-testing > >> PS C:\tmp> .\np-testing\Scripts\Activate.ps1 > >> (np-testing) PS C:\tmp> pip install -f > >> > <https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds>https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds > numpy nose > > > > > > C:\Python34\Scripts>pip installÂ "D:\Python > > distros\numpy-1.10.4-cp34-none-win_amd64.whl" > > Unpacking d:\python distros\numpy-1.10.4-cp34-none-win_amd64.whl > > Installing collected packages: numpy > > Successfully installed numpy > > Cleaning up... > > > > C:\Python34\Scripts>..\python > > Python 3.4.2 (v3.4.2:ab2c023a9432, OctÂ 6 > 2014, 22:16:31) [MSC v.1600 64 bit > > (AMD64)] on win32 > > Type "help", "copyright", "credits" or "license" for more information. > >>>> import numpy > >>>> numpy.test() > > Running unit tests for numpy > > NumPy version 1.10.4 > > NumPy relaxed strides checking option: False > > NumPy is installed in C:\Python34\lib\site-packages\numpy > > Python version 3.4.2 (v3.4.2:ab2c023a9432, OctÂ 6 2014, 22:16:31) [MSC > > v.1600 64 bit (AMD64)] > > nose version 1.3.7 > > > .......................F....S........................................................................................... > > > .....................................................................................................S.................. > > > ..........................................................................................C:\Python34\lib\unittest\case. > > py:162: DeprecationWarning: using a > non-integer number instead of an integer > > will result in an error in the future > >Â Â callable_obj(*args, **kwargs) > > ........C:\Python34\lib\unittest\case.py:162: DeprecationWarning: using a > > non-integer number instead of an integer will > > result in an error in the future > >Â Â callable_obj(*args, **kwargs) > > C:\Python34\lib\unittest\case.py:162: DeprecationWarning: using a > > non-integer number instead of an integer will result i > > n an error in the future > >Â Â callable_obj(*args, **kwargs) > > > .......................................................................................S................................ > > > ........................................................................................................................ > > > ..........................................................................C:\Python34\lib\unittest\case.py:162: > > Deprecat > > ionWarning: using a non-integer number instead of an integer will result in > > an error in the future > >Â Â callable_obj(*args, **kwargs) > > ..C:\Python34\lib\unittest\case.py:162: DeprecationWarning: using a > > non-integer number instead of an integer will result > >Â in an error in the future > >Â Â callable_obj(*args, **kwargs) > > C:\Python34\lib\unittest\case.py:162: DeprecationWarning: using a > > non-integer number instead of an integer will result i > > n an error in the future > >Â Â callable_obj(*args, **kwargs) > > C:\Python34\lib\unittest\case.py:162: DeprecationWarning: using a > > non-integer number instead of an integer will result i > > n an error in the future > >Â Â callable_obj(*args, **kwargs) > > C:\Python34\lib\unittest\case.py:162: DeprecationWarning: using a > > non-integer number instead of an integer will result i > > n an error in the future > >Â Â callable_obj(*args, **kwargs) > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ...............................................K.........................................................C:\Python34\lib > > \site-packages\numpy\ma\core.py:989: RuntimeWarning: invalid value > > encountered in multiply > >Â Â masked_da = umath.multiply(m, da) > > C:\Python34\lib\site-packages\numpy\ma\core.py:989: RuntimeWarning: invalid > > value encountered in multiply > >Â Â masked_da = umath.multiply(m, da) > > > ........................................................................................................................ > > > ..................C:\Python34\lib\site-packages\numpy\core\tests\test_numerictypes.py:372: > > DeprecationWarning: using a n > > on-integer number instead of an integer will result in an error in the > > future > >Â Â return self.ary['f0', 'f1'] > > > ........................................................................................................................ > > > .............................................................................................K.......................... > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ....K..K................................K...SK.S.......S................................................................ > > > ..................ES..SS................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ................................................................................S....................................... > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ...........................................K.................K.......................................................... > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ...........................................S............................................................................ > > > ..........................................S............................................................C:\Python34\lib\s > > ite-packages\numpy\ma\core.py:4089: UserWarning: Warning: converting a > > masked element to nan. > >Â Â warnings.warn("Warning: converting a masked element to nan.") > > > ..............................................................................................................C:\Python3 > > 4\lib\site-packages\numpy\ma\core.py:5116: RuntimeWarning: invalid value > > encountered in power > >Â Â np.power(out, 0.5, out=out, casting='unsafe') > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > > ........................................................................................................................ > > .......................... > > ====================================================================== > > ERROR: test_compile1 (test_system_info.TestSystemInfoReading) > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > >Â Â File > > "C:\Python34\lib\site-packages\numpy\distutils\tests\test_system_info.py", > > line 182, in test_compile1 > >Â Â Â c.compile([os.path.basename(self._src1)], output_dir=self._dir1) > >Â Â File "C:\Python34\lib\distutils\msvc9compiler.py", line 460, in compile > >Â Â Â self.initialize() > >Â Â File > "C:\Python34\lib\site-packages\numpy\distutils\msvccompiler.py", line > > 17, in initialize > >Â Â Â distutils.msvccompiler.MSVCCompiler.initialize(self, plat_name) > >Â Â File > "C:\Python34\lib\distutils\msvc9compiler.py", line 371, in initialize > >Â Â Â vc_env = query_vcvarsall(VERSION, plat_spec) > >Â Â File "C:\Python34\lib\distutils\msvc9compiler.py", line 259, in > > query_vcvarsall > >Â Â Â raise DistutilsPlatformError("Unable to find vcvarsall.bat") > > distutils.errors.DistutilsPlatformError: Unable to find vcvarsall.bat > > > > ====================================================================== > > FAIL: test_blasdot.test_blasdot_used > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > >Â Â File "C:\Python34\lib\site-packages\nose\case.py", line 198, in runTest > >Â Â Â self.test(*self.arg) > >Â Â File "C:\Python34\lib\site-packages\numpy\testing\decorators.py", line > > 146, in skipper_func > >Â Â Â return f(*args, **kwargs) > >Â Â File "C:\Python34\lib\site-packages\numpy\core\tests\test_blasdot.py", > > line 31, in test_blasdot_used > >Â Â Â assert_(dot is _dotblas.dot) > >Â Â File > "C:\Python34\lib\site-packages\numpy\testing\utils.py", line 53, in > > assert_ > >Â Â Â raise AssertionError(smsg) > > AssertionError > > > > ---------------------------------------------------------------------- > > Ran 5575 tests in 32.042s > > > > FAILED (KNOWNFAIL=8, SKIP=12, errors=1, failures=1) > > <nose.result.TextTestResult run=5575 errors=1 failures=1> > >Great - thanks - I got the same couple of failures - I believe they >are benign... > >Matthew >_______________________________________________ >NumPy-Discussion mailing list ><mailto:NumPy-Discussion@scipy.org>NumPy-Discussion(a)scipy.org >https://mail.scipy.org/mailman/listinfo/numpy-discussion > > >_______________________________________________ >NumPy-Discussion mailing list >NumPy-Discussion(a)scipy.org >https://mail.scipy.org/mailman/listinfo/numpy-discussion

2 1

Fwd: Windows wheels for testing
by Matthew Brett 12 Feb '16

12 Feb '16

On Fri, Feb 12, 2016 at 3:15 PM, R Schumacher <rays(a)blue-cove.com> wrote: > At 03:06 PM 2/12/2016, you wrote: > >> Any feedback would be very useful, > > > Sure, here's a little: > > C:\Python34\Scripts>pip install -f > https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds numpy > Requirement already satisfied (use --upgrade to upgrade): numpy in > c:\python34\lib\site-packages > Cleaning up... Thanks - yes - I should maybe have said that testing in a virtual environment is your best bet. Here's me testing from Powershell on a 32-bit Windows and on Pythons 3.5 and 2.7: PS C:\tmp> c:\Python35\python -m venv np-testing PS C:\tmp> .\np-testing\Scripts\Activate.ps1 (np-testing) PS C:\tmp> pip install -f https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds numpy nose Collecting numpy Downloading https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds/numpy-1.10.4-cp… (6.6MB) 100% |################################| 6.6MB 34kB/s Collecting nose Using cached nose-1.3.7-py3-none-any.whl Installing collected packages: numpy, nose Successfully installed nose-1.3.7 numpy-1.10.4 You are using pip version 7.1.2, however version 8.0.2 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command. (np-testing) PS C:\tmp> python -c 'import numpy; numpy.test()' Running unit tests for numpy NumPy version 1.10.4 NumPy relaxed strides checking option: False NumPy is installed in C:\tmp\np-testing\lib\site-packages\numpy Python version 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:16:59) [MSC v.1900 32 bit (Intel)] nose version 1.3.7 ......... [test output] Now on 2.7 (np-testing) PS C:\tmp> deactivate PS C:\tmp> C:\Python27\Scripts\pip.exe install virtualenv [...] output snipped PS C:\tmp> C:\Python27\Scripts\virtualenv.exe np27-testing Ignoring indexes: https://pypi.python.org/simple Collecting pip Collecting wheel Collecting setuptools Installing collected packages: pip, wheel, setuptools Successfully installed pip-7.1.2 setuptools-18.5 wheel-0.26.0 PS C:\tmp> .\np27-testing\Scripts\activate.ps1 (np27-testing) PS C:\tmp> pip install -f https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds numpy nose Collecting numpy c:\tmp\np27-testing\lib\site-packages\pip\_vendor\requests\packages\urllib3\util\ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatf ormwarning. InsecurePlatformWarning Downloading https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds/numpy-1.10.4-cp… (6.4MB) 100% |################################| 6.4MB 35kB/s Collecting nose c:\tmp\np27-testing\lib\site-packages\pip\_vendor\requests\packages\urllib3\util\ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatf ormwarning. InsecurePlatformWarning Using cached nose-1.3.7-py2-none-any.whl Installing collected packages: numpy, nose Successfully installed nose-1.3.7 numpy-1.10.4 You are using pip version 7.1.2, however version 8.0.2 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command. (np27-testing) PS C:\tmp> python -c 'import numpy; numpy.test()' Running unit tests for numpy NumPy version 1.10.4 NumPy relaxed strides checking option: False NumPy is installed in c:\tmp\np27-testing\lib\site-packages\numpy Python version 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] nose version 1.3.7 .............. [etc] Thanks a lot for testing, Matthew

1 0

Re: [Numpy-discussion] Windows wheels for testing
by R Schumacher 12 Feb '16

12 Feb '16

At 03:06 PM 2/12/2016, you wrote: >Any feedback would be very useful, Sure, here's a little: C:\Python34\Scripts>pip install -f https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds numpy Requirement already satisfied (use --upgrade to upgrade): numpy in c:\python34\lib\site-packages Cleaning up...

1 0

Windows wheels for testing
by Matthew Brett 12 Feb '16

12 Feb '16

Hi, We're talking about putting up Windows wheels for numpy on pypi - here: https://github.com/numpy/numpy/issues/5479 I've built some wheels that might be suitable - available here: http://nipy.bic.berkeley.edu/scipy_installers/atlas_builds/ I'd be very grateful if y'all would test these. They should install with something like: pip install -f https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds numpy This should work for Pythons 2.7, 3.4, 3.5, both 32 and 64-bit. Any feedback would be very useful, Cheers, Matthew

1 0

Hook in __init__.py to let distributors patch numpy
by Matthew Brett 12 Feb '16

12 Feb '16

Hi, Over at https://github.com/numpy/numpy/issues/5479 we're discussing Windows wheels. On thing that we would like to be able to ship Windows wheels, is to be able to put some custom checks into numpy when you build the wheels. Specifically, for Windows, we're building on top of ATLAS BLAS / LAPACK, and we need to check that the system on which the wheel is running, has SSE2 instructions, otherwise we know ATLAS will crash (almost everybody does have SSE2 these days). The way I propose we do that, is this patch here: https://github.com/numpy/numpy/pull/7231 diff --git a/numpy/__init__.py b/numpy/__init__.py index 0fcd509..ba3ba16 100644 --- a/numpy/__init__.py +++ b/numpy/__init__.py @@ -190,6 +190,12 @@ def pkgload(*packages, **options): test = testing.nosetester._numpy_tester().test bench = testing.nosetester._numpy_tester().bench + # Allow platform-specific build to intervene in numpy init + try: + from . import _distributor_init + except ImportError: + pass + from . import core from .core import * from . import compat So, numpy __init__.py looks for a module `_distributor_init`, in which the distributor might have put custom code to do any checks and initialization needed for the particular platform. We don't by default ship a `_distributor_init.py` but leave it up to packagers to generate this when building binaries. Does that sound like a sensible approach to y'all? Cheers, Matthew

3 2

Deprecating `numpy.iterable`
by Joseph Fox-Rabinovitz 11 Feb '16

11 Feb '16

I have created a PR to deprecate `np.iterable` (https://github.com/numpy/numpy/pull/7202). It is a very old function, introduced as a utility in 2005 (https://github.com/numpy/numpy/commit/052a7b2e3276a303be1083022fc24d43084d2…), and there is no good reason for it to be part of the public API. It is used internally 10 times within numpy. I have repaced those usages with a private function `np.lib.function_base._iterable` and added a `DeprecationWarning` to the public function. Is there anyone that objects to deprecating this function? Regards, -Joseph

4 4

Build fortran extension on Windows with gfortran and MSVC
by Marek Wojciechowski 11 Feb '16

11 Feb '16

Hi! It seems that on Windows + Python-3.5 fortran extensions cannot be built anymore with f2py and mingw32 compilers, because of new MSVC. Here is the short description of the errors one gets: http://stackoverflow.com/questions/33822554/build-fortran-extension-on-wind… This is sad, because this worked quite nicely with all previous python versions. I'm curious what are the options now? Is this going to be supported in the future, with mingw-w64 for example? Regards, -- Marek --- Politechnika ÅÃ³dzka Lodz University of Technology TreÅÄ tej wiadomoÅci zawiera informacje przeznaczone tylko dla adresata. JeÅ¼eli nie jesteÅcie PaÅstwo jej adresatem, bÄdÅº otrzymaliÅcie jÄ przez pomyÅkÄ prosimy o powiadomienie o tym nadawcy oraz trwaÅe jej usuniÄcie. This email contains information intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient or if you have received this message in error, please notify the sender and delete it from your system.

2 3

Linking other libm-Implementation
by Nils Becker 10 Feb '16

10 Feb '16

Hi all, I wanted to know if there is any sane way to build numpy while linking to a different implementation of libm? A drop-in replacement for libm (e.g. openlibm) should in principle work, I guess, but I did not manage to actually make it work. As far as I understand the build code, setting MATHLIB=openlibm should suffice, but it did not. The build works fine, but in the end when running numpy apparently the functions of the system libm.so are used. I could not verify this directly (as I do not know how) but noticed that there is no performance difference between the builds - while there is one with pure C programs linked against libm and openlibm. Using amdlibm would require some work as the functions are prefixed with "_amd", I guess? Using intels libimf should work when using intels compiler, but I did not try this. With gcc I did not get it to work. A quite general question: At the moment the performance and the accuracy of the base mathematical functions depends on the platform and libm-Implementation of the system. Although there are functions defined in npy_math, they are only used as fall-backs, if they are not provided by a library. (correct me if I am wrong here) Is there some plan to change this in the future and provide defined behaviour (specified accuracy and/or speed) across platforms? As I understood it Julia started openlibm for this reason (which is based on fdlibm/msun, same as npy_math). Cheers Nils

6 10

Numpy 1.11.0b2 released
by Charles R Harris 10 Feb '16

10 Feb '16

Hi All, I hope I am pleased to announce the Numpy 1.11.0b2 release. The first beta was a damp squib due to missing files in the released source files, this release fixes that. The new source filese may be downloaded from sourceforge, no binaries will be released until the mingw tool chain problems are sorted. Please test and report any problem. Chuck

17 43

Fwd: Multi-distribution Linux wheels - please test
by Evgeni Burovski 09 Feb '16

09 Feb '16

---------- Forwarded message ---------- From: Evgeni Burovski <evgeny.burovskiy(a)gmail.com> Date: Mon, Feb 8, 2016 at 11:56 AM Subject: Re: [Numpy-discussion] Multi-distribution Linux wheels - please test To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> On Sat, Feb 6, 2016 at 8:26 PM, Matthew Brett <matthew.brett(a)gmail.com> wrote: > Hi, > > As some of you may have seen, Robert McGibbon and Nathaniel have just > guided a PEP for multi-distribution Linux wheels past the approval > process over on distutils-sig: > > https://www.python.org/dev/peps/pep-0513/ > > The PEP includes a docker image on which y'all can build wheels which > match the PEP: > > https://quay.io/repository/manylinux/manylinux > > Now we're at the stage where we need stress-testing of the built > wheels to find any problems we hadn't thought of. > > I've built numpy and scipy wheels here: > > https://nipy.bic.berkeley.edu/manylinux/ > > So, if you have a Linux distribution handy, we would love to hear from > you about the results of testing these guys, maybe on the lines of: > > pip install -f https://nipy.bic.berkeley.edu/manylinux numpy scipy > python -c 'import numpy; numpy.test()' > python -c 'import scipy; scipy.test()' > > These manylinux wheels should soon be available on pypi, and soon > after, installable with latest pip, so we would like to fix as many > problems as possible before going live. > > Cheers, > > Matthew > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion Hi, Bog-standard Ubuntu 12.04, fresh virtualenv: Python 2.7.3 (default, Jun 22 2015, 19:33:41) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.10.4' >>> numpy.test() Running unit tests for numpy NumPy version 1.10.4 NumPy relaxed strides checking option: False NumPy is installed in /home/br/virtualenvs/manylinux/local/lib/python2.7/site-packages/numpy Python version 2.7.3 (default, Jun 22 2015, 19:33:41) [GCC 4.6.3] nose version 1.3.7 <snip> ====================================================================== ERROR: test_multiarray.TestNewBufferProtocol.test_relaxed_strides ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/br/virtualenvs/manylinux/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/br/virtualenvs/manylinux/local/lib/python2.7/site-packages/numpy/core/tests/test_multiarray.py", line 5366, in test_relaxed_strides fd.write(c.data) TypeError: 'buffer' does not have the buffer interface ---------------------------------------------------------------------- * Scipy tests pass with one error in TestNanFuncs, but the interpreter crashes immediately afterwards. Same machine, python 3.5: both numpy and scipy tests pass.

4 13