Hi all, I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags): <traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in <bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int16)> ignored </traceback> I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not. Thanks!
Hi, Has anyone confirmed if this is a bug? Should I post this in the bug tracker? Thanks! On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes <totonixsame@gmail.com> wrote:
Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in <bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int16)> ignored </traceback>
I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes <totonixsame@gmail.com> wrote:
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug? Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes <totonixsame@gmail.com> wrote:
Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in <bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int16)> ignored </traceback>
I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Apr 26, 2011 at 4:21 PM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes <totonixsame@gmail.com> wrote: tracker?
I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug?
Just a confirmation: I get the error with numpy 1.5.1, Python 2.7.1 in Windows XP, but not in Mac OSX or Linux (Ubuntu). Warren Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes <totonixsame@gmail.com> wrote:
Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in
_bootstrap
self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in <bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int16)> ignored </traceback>
I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Apr 26, 2011 at 6:21 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes <totonixsame@gmail.com> wrote:
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug?
Ralf
Hi Ralf, Done http://projects.scipy.org/numpy/ticket/1809 Thanks!
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes <totonixsame@gmail.com> wrote:
Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in <bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int16)> ignored </traceback>
I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I don't think this was working correctly in numpy 1.4 either. The underlying problem seems to be that instance attributes of ndarray subtypes get lost during pickling: import pickle import numpy as np class aarray(np.ndarray): def __new__(subtype): self = np.ndarray.__new__(subtype, (1,)) self.attr = 'attr' return self def __array_finalize__(self, obj): self.attr = getattr(obj, 'attr', None) a = aarray() b = pickle.loads(a.dumps()) assert a.attr == b.attr, (a.attr, b.attr) AssertionError: ('attr', None) Christoph On 4/26/2011 2:21 PM, Ralf Gommers wrote:
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes <totonixsame@gmail.com> wrote:
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug?
Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes <totonixsame@gmail.com> wrote:
Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between<traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in<bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int16)> ignored </traceback>
I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Apr 27, 2011 at 4:07 PM, Christoph Gohlke <cgohlke@uci.edu> wrote:
I don't think this was working correctly in numpy 1.4 either. The underlying problem seems to be that instance attributes of ndarray subtypes get lost during pickling:
import pickle import numpy as np class aarray(np.ndarray): def __new__(subtype): self = np.ndarray.__new__(subtype, (1,)) self.attr = 'attr' return self def __array_finalize__(self, obj): self.attr = getattr(obj, 'attr', None) a = aarray() b = pickle.loads(a.dumps()) assert a.attr == b.attr, (a.attr, b.attr)
AssertionError: ('attr', None)
Christoph
possibly related to this ticket: http://projects.scipy.org/numpy/ticket/1452
On 4/26/2011 2:21 PM, Ralf Gommers wrote:
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes <totonixsame@gmail.com> wrote:
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug?
Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes <totonixsame@gmail.com> wrote:
Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between<traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in<bound method memmap.__del__ of memmap([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype=int16)> ignored </traceback>
I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (6)
-
Brent Pedersen -
Christoph Gohlke -
Ralf Gommers -
Thiago Franco de Moraes -
Thiago Franco Moraes
-
Warren Weckesser