Hi all,
I'm having a error using memmap objects shared among processes created
by the multprocessing module. This error only happen in Windows with
numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac
OS X it doesn't happen. This error is demonstrated by this little
example script here https://gist.github.com/929168 , and the traceback
is bellow (between <traceback> tags):
<traceback>
Process Process-1:
Traceback (most recent call last):
File "C:\Python26\Lib\multiprocessing\process.py", line 232, in
_bootstrap
self.run()
File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run
self._target(*self._args, **self._kwargs)
File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in
print_ma
trix
print matrix
File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line
1379, in arra
y_str
return array2string(a, max_line_width, precision, suppress_small, '
', "", s
tr)
File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line
309, in ar
ray2string
separator, prefix)
File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line
189, in _a
rray2string
data = _leading_trailing(a)
File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line
162, in _l
eading_trailing
min(len(a), _summaryEdgeItems))]
File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257,
in __arra
y_finalize__
self.filename = obj.filename
AttributeError: 'memmap' object has no attribute 'filename'
Exception AttributeError: AttributeError("'NoneType' object has no
attribute 'te
ll'",) in
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes
Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in
ignored </traceback> I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug? Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes
wrote: Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in
ignored </traceback> I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Apr 26, 2011 at 4:21 PM, Ralf Gommers
Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes
wrote: tracker? I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug?
Just a confirmation: I get the error with numpy 1.5.1, Python 2.7.1 in Windows XP, but not in Mac OSX or Linux (Ubuntu). Warren Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes
wrote: Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in
_bootstrap
self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in
ignored </traceback> I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Apr 26, 2011 at 6:21 PM, Ralf Gommers
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes
wrote: Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug?
Ralf
Hi Ralf, Done http://projects.scipy.org/numpy/ticket/1809 Thanks!
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes
wrote: Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between <traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in
ignored </traceback> I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I don't think this was working correctly in numpy 1.4 either. The underlying problem seems to be that instance attributes of ndarray subtypes get lost during pickling: import pickle import numpy as np class aarray(np.ndarray): def __new__(subtype): self = np.ndarray.__new__(subtype, (1,)) self.attr = 'attr' return self def __array_finalize__(self, obj): self.attr = getattr(obj, 'attr', None) a = aarray() b = pickle.loads(a.dumps()) assert a.attr == b.attr, (a.attr, b.attr) AssertionError: ('attr', None) Christoph On 4/26/2011 2:21 PM, Ralf Gommers wrote:
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes
wrote: Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug?
Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes
wrote: Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between<traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in
ignored </traceback> I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Apr 27, 2011 at 4:07 PM, Christoph Gohlke
I don't think this was working correctly in numpy 1.4 either. The underlying problem seems to be that instance attributes of ndarray subtypes get lost during pickling:
import pickle import numpy as np class aarray(np.ndarray): def __new__(subtype): self = np.ndarray.__new__(subtype, (1,)) self.attr = 'attr' return self def __array_finalize__(self, obj): self.attr = getattr(obj, 'attr', None) a = aarray() b = pickle.loads(a.dumps()) assert a.attr == b.attr, (a.attr, b.attr)
AssertionError: ('attr', None)
Christoph
possibly related to this ticket: http://projects.scipy.org/numpy/ticket/1452
On 4/26/2011 2:21 PM, Ralf Gommers wrote:
On Mon, Apr 25, 2011 at 1:16 PM, Thiago Franco Moraes
wrote: Hi,
Has anyone confirmed if this is a bug? Should I post this in the bug tracker?
I see the same thing with recent master. Something very strange is going on in the memmap.__array_finalize__ method under Windows. Can you file a bug?
Ralf
Thanks!
On Tue, Apr 19, 2011 at 9:01 PM, Thiago Franco de Moraes
wrote: Hi all,
I'm having a error using memmap objects shared among processes created by the multprocessing module. This error only happen in Windows with numpy 1.5 or above, in numpy 1.4.1 it doesn't happen, In Linux and Mac OS X it doesn't happen. This error is demonstrated by this little example script here https://gist.github.com/929168 , and the traceback is bellow (between<traceback> tags):
<traceback> Process Process-1: Traceback (most recent call last): File "C:\Python26\Lib\multiprocessing\process.py", line 232, in _bootstrap self.run() File "C:\Python26\Lib\multiprocessing\process.py", line 88, in run self._target(*self._args, **self._kwargs) File "C:\Documents and Settings\phamorim\Desktop\test.py", line 7, in print_ma trix print matrix File "C:\Python26\Lib\site-packages\numpy\core\numeric.py", line 1379, in arra y_str return array2string(a, max_line_width, precision, suppress_small, ' ', "", s tr) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 309, in ar ray2string separator, prefix) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 189, in _a rray2string data = _leading_trailing(a) File "C:\Python26\Lib\site-packages\numpy\core\arrayprint.py", line 162, in _l eading_trailing min(len(a), _summaryEdgeItems))] File "C:\Python26\Lib\site-packages\numpy\core\memmap.py", line 257, in __arra y_finalize__ self.filename = obj.filename AttributeError: 'memmap' object has no attribute 'filename' Exception AttributeError: AttributeError("'NoneType' object has no attribute 'te ll'",) in
ignored </traceback> I don't know if it's a bug, but I thought it's is import to report because the version 1.4.1 was working and 1.5.0 and above was not.
Thanks!
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (6)
-
Brent Pedersen
-
Christoph Gohlke
-
Ralf Gommers
-
Thiago Franco de Moraes
-
Thiago Franco Moraes
-
Warren Weckesser