
Memmapped arrays don't pickle right. I know that to get them to really pickle and restore identically, we would need some effort. However, in the current status, pickling and restoring a memmapped array leads to tracebacks that seem like they could be avoided. I am attaching a patch with a test that shows the problem, and a fix. Should I create a ticket, or is this light-enough to be applied immediatly? Cheers, Gaël

On Mon, May 24, 2010 at 3:25 PM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
Memmapped arrays don't pickle right. I know that to get them to really pickle and restore identically, we would need some effort. However, in the current status, pickling and restoring a memmapped array leads to tracebacks that seem like they could be avoided.
I am attaching a patch with a test that shows the problem, and a fix. Should I create a ticket, or is this light-enough to be applied immediatly?
Cheers,
Gaël
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
also check this: http://projects.scipy.org/numpy/ticket/1452 still needs work.

On Mon, May 24, 2010 at 03:33:09PM -0700, Brent Pedersen wrote:
On Mon, May 24, 2010 at 3:25 PM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
Memmapped arrays don't pickle right. I know that to get them to really pickle and restore identically, we would need some effort. However, in the current status, pickling and restoring a memmapped array leads to tracebacks that seem like they could be avoided.
I am attaching a patch with a test that shows the problem, and a fix. Should I create a ticket, or is this light-enough to be applied immediatly?
also check this: http://projects.scipy.org/numpy/ticket/1452
still needs work.
Does look good. Is there an ETA for your patch to be applied? Right now this bug is making code crash when memmapped arrays are used (eg multiprocessing), so a hot fix can be useful, without removing any merit to your work that addresses the underlying problem. Cheers, Gaël

On Mon, May 24, 2010 at 3:37 PM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
On Mon, May 24, 2010 at 03:33:09PM -0700, Brent Pedersen wrote:
On Mon, May 24, 2010 at 3:25 PM, Gael Varoquaux <gael.varoquaux@normalesup.org> wrote:
Memmapped arrays don't pickle right. I know that to get them to really pickle and restore identically, we would need some effort. However, in the current status, pickling and restoring a memmapped array leads to tracebacks that seem like they could be avoided.
I am attaching a patch with a test that shows the problem, and a fix. Should I create a ticket, or is this light-enough to be applied immediatly?
also check this: http://projects.scipy.org/numpy/ticket/1452
still needs work.
Does look good. Is there an ETA for your patch to be applied?
Right now this bug is making code crash when memmapped arrays are used (eg multiprocessing), so a hot fix can be useful, without removing any merit to your work that addresses the underlying problem.
Cheers,
Gaël _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
gael, not sure about ETA of application. i think the main remaining problem (other than more tests) is py3 support--as charris points out in the ticket. i have a start which shadows numpy's __getitem__, but havent fixed all the bugs--and not sure that's a good idea. my original patch was quite simple as well, but once it starts supporting all versions and more edge cases ...
participants (2)
-
Brent Pedersen
-
Gael Varoquaux