[Numpy-discussion] Python crashes while printing a huge numpy array (Windows 1.6b2)

[Charles R Harris]

On Sat, Apr 23, 2011 at 3:47 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
> On Sat, Apr 23, 2011 at 3:38 PM, Christoph Gohlke <cgohlke at uci.edu> wrote:
>
>>
>>
>> On 4/23/2011 10:41 AM, Charles R Harris wrote:
>> >
>> >
>> > On Sat, Apr 23, 2011 at 11:09 AM, Bruce Southey <bsouthey at gmail.com
>> > <mailto:bsouthey at gmail.com>> wrote:
>> >
>> >     On Sat, Apr 23, 2011 at 9:58 AM, Till Stensitzki <mail.till at gmx.de
>> >     <mailto:mail.till at gmx.de>> wrote:
>> >     >
>> >     > > Do you also have an earlier version of numpy installed? As David
>> >     says, this
>> >     > >should raise an error for recent numpy and
>> >     > >I'm wondering if you are inadvertently
>> >     > >running an earlier version.Chuck
>> >     >
>> >     >
>> >     >  I only have one python installation and
>> >     >  numpy.__version__ shows 1.6b.
>> >     >  I could reinstall numpy, if it would help.
>> >     >
>> >     >  _______________________________________________
>> >     >  NumPy-Discussion mailing list
>> >     >  NumPy-Discussion at scipy.org <mailto:NumPy-Discussion at scipy.org>
>> >     >  http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >     >
>> >
>> >     Hi,
>> >     I can get this with 64-bit Win 7, 32-bit Python 2.6, 2.7 (below) and
>> >     3.1 and numpy 1.6b (fresh install)  IDLE and the command line. I can
>> >     also confirm the 'ValueError' with Python2.6 and numpy 1.51 on the
>> >     same system.
>> >
>> >     Actually this is 'weird' when printing and crashed with the range -
>> >     accessing unassigned memory?
>> >     A smaller array gives an numpy error or memory error in idle.
>> >
>> >     Bruce
>> >
>> >
>> >     > >> import numpy as np
>> >     > >> x=np.zeros((262144, 262144))
>> >     > >> x
>> >     array([], shape=(262144, 262144), dtype=float64)
>> >     > >> x[0,0]
>> >     2.1453735050108555e-314
>> >     > >> x[1:10,1:10]
>> >
>> >     > >> ================================ RESTART
>> >     ================================
>> >     > >> import numpy as np
>> >
>> >     > >> x=np.zeros((26214, 26214))
>> >
>> >     Traceback (most recent call last):
>> >       File "<pyshell#7>", line 1, in <module>
>> >         x=np.zeros((26214, 26214))
>> >     ValueError: array is too big.
>> >     > >>
>> >     > >> x=np.zeros((262144, 26214))
>> >
>> >     Traceback (most recent call last):
>> >       File "<pyshell#8>", line 1, in <module>
>> >         x=np.zeros((262144, 26214))
>> >     MemoryError
>> >     _____
>> >
>> >
>> > This was fixed before, maybe it got broken again. Since this looks
>> > windows specific, I'm guessing it has something to do with the size of
>> > long being 32 bits.
>> >
>> > The previous problem was integer overflow when multiplying the
>> > dimensions together to get the array size when repeated divisions of the
>> > maximum size should have be used instead.
>> >
>> > Chuck
>>
>> Could be related to this change:
>>
>> <
>> https://github.com/numpy/numpy/commit/fcc6cc73ddcb1fc85446ba9256ac24ecdda6c6d8#L1L1121
>> >
>>
>>
> My, that does look suspicious ;) Could you revert that loop and test it
> out?
>
> There was also a function for doing that check, I don't recall which, and
> it should probably be checked to make sure it remains as was.
>
>
There was also a test that should have caught this. Evidently it wasn't
quite up to snuff.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110423/e3c781d8/attachment.html>