[Numpy-discussion] lazy loading ndarrays

[Joe Kington]

Similar to what Matthew said, I often find that it's cleaner to make a
seperate class with a "data" (or somesuch) property that lazily loads the
numpy array.

For example, something like:

class DataFormat(object):
    def __init__(self, filename):
        self.filename = filename
        for key, value in self._read_header().iteritems():
            setattr(self, key, value)

    @property
    def data(self):
        try:
            return self._data
        except AttributeError:
            self._data = self._read_data()
            return self._data

Hope that helps,
-Joe

On Tue, Jul 26, 2011 at 4:15 PM, Matthew Brett <matthew.brett at gmail.com>wrote:

> Hi,
>
> On Tue, Jul 26, 2011 at 5:11 PM, Craig Yoshioka <craigyk at me.com> wrote:
> > I want to subclass ndarray to create a class for image and volume data,
> and when referencing a file I'd like to have it load the data only when
> accessed.  That way the class can be used to quickly set and manipulate
> header values, and won't load data unless necessary.  What is the best way
> to do this?  Are there any hooks I can use to load the data when an array's
> values are first accessed or manipulated?  I tried some trickery with
> __array_interface__ but couldn't get it to work very well.  Should I just
> use a memmapped array, and give up on a purely 'lazy' approach?
>
> What kind of images are you loading?   We do lazy loading in nibabel,
> for medical image type formats:
>
> http://nipy.sourceforge.net/nibabel/
>
> - but our images _have_ arrays and headers, rather than (appearing to
> be) arrays.  Thus something like:
>
> import nibabel as nib
>
> img = nib.load('my_image.img')
> # data not loaded at this point
> data = img.get_data()
> # data loaded now.  Maybe memmapped if the format allows
>
> If you think you might have similar needs, I'd be very happy to help
> you get going in nibabel...
>
> Best,
>
> Matthew
>  _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20110726/4b2bf51b/attachment.html>