[Numpy-discussion] multivariate_normal issue with 'size' argument

Christoph Gohlke cgohlke at uci.edu
Fri May 24 15:09:21 EDT 2013


On 5/24/2013 7:00 AM, josef.pktd at gmail.com wrote:
> On Fri, May 24, 2013 at 9:47 AM, Warren Weckesser
> <warren.weckesser at gmail.com> wrote:
>> On 5/24/13, Peter Cock <p.j.a.cock at googlemail.com> wrote:
>>> On Fri, May 24, 2013 at 2:15 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>>> On Fri, May 24, 2013 at 9:12 AM, Peter Cock <p.j.a.cock at googlemail.com>
>>>> wrote:
>>>>> On Fri, May 24, 2013 at 1:59 PM, Emanuele Olivetti
>>>>> <emanuele at relativita.com> wrote:
>>>>>> Interesting. Anyone able to reproduce what I observe?
>>>>>>
>>>>>> Emanuele
>>>>>
>>>>>
>>>>> Yes, I can reproduce this IndexError under Mac OS X:
>>>>>
>>>>> $ which python2.7
>>>>> /usr/bin/python2.7
>>>>> $ python2.7
>>>>> Python 2.7.2 (default, Oct 11 2012, 20:14:37)
>>>>> [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on
>>>>> darwin
>>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>
>>>> Can everyone please report whether they have a 32-bit build of Python
>>>> or a 64-bit build? That's probably the most relevant factor.
>>>
>>> It seems to affect all of 32 bit Windows XP, 64 bit Mac, 32 bit Linux,
>>> and 64 bit Linux
>>> for some versions of NumPy...  Thus far the only non-failure I've seen
>>> is 64 bit Linux,
>>> Python 2.6.6 with NumPy 1.6.2 (other Python/NumPy installs on this
>>> machine do fail).
>>>
>>> Its a bit strange - I don't see any obvious pattern.
>>>
>>> Peter
>>>
>>> -------------------------------------------
>>>
>>> Failures:
>>>
>>> My Python installs on this Mac all seem to be 64bit (and fail),
>>>
>>> $ python3.3
>>> Python 3.3.1 (default, Apr  8 2013, 17:54:08)
>>> [GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.57))] on
>>> darwin
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffffffffffff True
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[ 1.80932387  0.85894164]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 4161, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:19140)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.7.1'
>>>>>> quit()
>>>
>>> This also affects NumPy 1.5 so this isn't a recent regression:
>>>
>>> $ python3.2
>>> Python 3.2 (r32:88445, Feb 28 2011, 17:04:33)
>>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffffffffffff True
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[ 1.11403341 -1.67856405]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 3954, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:17234)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.5.0'
>>>
>>> $ python3.1
>>> Python 3.1.2 (r312:79147, Nov 15 2010, 16:28:52)
>>> [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffffffffffff True
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[ 0.3834108  -0.31124203]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 3954, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:17234)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.5.0'
>>>>>> quit()
>>>
>>> And on my 32 bit Windows XP box,
>>>
>>> Python 2.7 (r27:82525, Jul  4 2010, 09:01:59) [MSC v.1500 32 bit
>>> (Intel)] on win32
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> ('7fffffff', False)
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.35072523 -0.58046885]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 3954, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy\random\mtrand\mtrand.c:17234)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.5.0'
>>>>>>
>>>
>>> Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
>>> 32 bit (Intel)] on win32
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.00453374  0.2210342 ]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 4142, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy\random\mtrand\mtrand.c:19128)
>>> IndexError: invalid index to scalar variable.
>>>>>>
>>>>>> np.__version__
>>> '1.7.0rc2'
>>>
>>> Here's a couple of runs from an old 32 bit Linux machine which also
>>> shows the problem:
>>>
>>> $ python2.7
>>> Python 2.7 (r27:82500, Nov 12 2010, 14:19:08)
>>> [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> ('7fffffff', False)
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[ 0.40887932 -0.09872295]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 3952, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:17027)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.4.1'
>>>>>> quit()
>>>
>>> $ python3.3
>>> Python 3.3.0 (default, Oct 31 2012, 13:19:08)
>>> [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffff False
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.16796654 -1.71312415]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 4127, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:18657)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.7.0b2'
>>>>>> quit()
>>>
>>> And a more up to date 64 bit Linux machine where this fails on
>>> some versions of NumPy:
>>>
>>> $ python3.3
>>> Python 3.3.1 (default, Apr  8 2013, 14:38:06)
>>> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> 7fffffffffffffff True
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.18862683 -0.92177665]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File "mtrand.pyx", line 4127, in
>>> mtrand.RandomState.multivariate_normal
>>> (numpy/random/mtrand/mtrand.c:18657)
>>> IndexError: invalid index to scalar variable.
>>>>>> np.__version__
>>> '1.7.0b2'
>>>>>> quit()
>>>
>>> -------------------------------------------
>>> Successes
>>> -------------------------------------------
>>>
>>> 64 bit Linux:
>>>
>>> $ python2.6
>>> Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
>>> [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> import sys;print("%x" % sys.maxsize, sys.maxsize > 2**32)
>>> ('7fffffffffffffff', True)
>>>>>> import numpy as np
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=1))
>>> [[-0.27469218 -2.12911784]]
>>>>>> print(np.random.multivariate_normal(mean=np.zeros(2), cov=np.eye(2),
>>>>>> size=np.int64(1)))
>>> [[ 0.02609307  0.32485211]]
>>>>>> np.__version__
>>> '1.6.2'
>>>>>> quit()
>>>
>>>
>>
>>
>> Peter: wow, that's a lot of tests!
>>
>> Two more data points:
>> On Ubuntu 12.04, using 64 bit builds of Python 2.7.4 (from Anaconda
>> 1.5.0), and numpy built from source: numpy 1.6.1 gives the error, but
>> 1.6.2 does not.
>
> I also get them on Windows 64 (gohlke binaries)
> python 3.2 with numpy 1.6.1
> python 3.3 with numpy '1.7.0rc1'
>
> Josef
>>
>> Warren
>>
>>
>>>
>>> Peter


This patch works for me:

diff --git a/numpy/random/mtrand/mtrand.pyx b/numpy/random/mtrand/mtrand.pyx
index b0de560..233ff52 100644
--- a/numpy/random/mtrand/mtrand.pyx
+++ b/numpy/random/mtrand/mtrand.pyx
@@ -4156,7 +4156,7 @@ cdef class RandomState:
          if mean.shape[0] != cov.shape[0]:
                 raise ValueError("mean and cov must have same length")
          # Compute shape of output
-        if isinstance(shape, int):
+        if isinstance(shape, (int, long, np.integer)):
              shape = [shape]
          final_shape = list(shape[:])
          final_shape.append(mean.shape[0])

--
Christoph



More information about the NumPy-Discussion mailing list