[pypy-dev] Memap in numpypy

Mike Beller mike at tradeworx.com
Fri Jun 14 16:26:02 CEST 2013


Hello

Do you think it is likely that the memap capabilities of numpy will find
their way in to numpypy any time soon?

It seems to me that some people think memap is a relatively unimportant
aspect of numpy.  But I do not think so.  Because of the way the linux IO
subsystem and virtual memory systems interact, memap enables numpy to have
high performance access to very large data sets -- it helps make numpy
relevant to "Big Data".

The code to allow numpy to support memap doesn't seem very large.  But,
while I have tried reading through the code, I really can not tell whether
the same is true for numpypy, or whether it is a large endeavor (for
example, due to some kind of pypy memory management architectural issue).

I'm interested in any input on this.

Mike Beller
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130614/357757d4/attachment.html>


More information about the pypy-dev mailing list