On Wed, 2003-07-16 at 21:13, Dave Cole wrote:
"Todd" == Todd Miller
writes: Todd> I think however that memory mapping sequences or arbitrary Todd> Python objects isn't going to happen in numarray any time soon; Todd> it sounds too much like object persistence.
It is used to minimize the cost of huge datasets.
That's what we use it for too... but only for primitive data types.
I know that is a big ask! We have an extension module which stores variable length blobs in a single memory-mapped file which might be useful - the code could be made available to the numarray project, I think.
Todd> I don't understand the difference between your module and Todd> Python's mmap.
The code could use probably the Python mmap module. It would be difficult because all access to the Python mmap object is via Python.
A CObject exported API in the Python mmap module would be very useful.
What do you want to get via the CObject API? Since any mmap can be treated as a buffer, it's easy to get a pointer and size via the Python buffer API in C. Is there something else you need?
- Dave -- Todd Miller