[Numpy-discussion] Tracking and inspecting numpy objects

[Eelco Hoogendoorn]

I figured the reference to the object through the local scope would also be
tracked by the GC somehow, since the entire stack frame can be regarded as
a separate object itself, but apparently not.

On Mon, Sep 15, 2014 at 1:06 PM, Mads Ipsen <mads.ipsen at gmail.com> wrote:

> Thanks to everybody for taking time to answer!
>
> Best regards,
>
> Mads
>
> On 15/09/14 12:11, Sebastian Berg wrote:
> > On Mo, 2014-09-15 at 12:05 +0200, Eelco Hoogendoorn wrote:
> >>
> >>
> >>
> >> On Mon, Sep 15, 2014 at 11:55 AM, Sebastian Berg
> >> <sebastian at sipsolutions.net> wrote:
> >>          On Mo, 2014-09-15 at 10:16 +0200, Mads Ipsen wrote:
> >>          > Hi,
> >>          >
> >>          > I am trying to inspect the reference count of numpy arrays
> >>          generated by
> >>          > my application.
> >>          >
> >>          > Initially, I thought I could inspect the tracked objects
> >>          using
> >>          > gc.get_objects(), but, with respect to numpy objects, the
> >>          returned
> >>          > container is empty. For example:
> >>          >
> >>          > import numpy
> >>          > import gc
> >>          >
> >>          > data = numpy.ones(1024).reshape((32,32))
> >>          >
> >>          > objs = [o for o in gc.get_objects() if isinstance(o,
> >>          numpy.ndarray)]
> >>          >
> >>          > print objs                # Prints empty list
> >>          > print gc.is_tracked(data) # Print False
> >>          >
> >>          > Why is this? Also, is there some other technique I can use
> >>          to inspect
> >>          > all numpy generated objects?
> >>          >
> >>
> >>          Two reasons. First of all, unless your array is an object
> >>          arrays (or a
> >>          structured one with objects in it), there are no objects to
> >>          track. The
> >>          array is a single python object without any referenced objects
> >>          (except
> >>          possibly its `arr.base`).
> >>
> >>          Second of all -- and this is an issue -- numpy doesn't
> >>          actually
> >>          implement the traverse slot, so it won't even work for object
> >>          arrays
> >>          (numpy object arrays do not support circular garbage
> >>          collection at this
> >>          time, please feel free to implement it ;)).
> >>
> >>          - Sebastian
> >>
> >>
> >>
> >>
> >>
> >>
> >> Does this answer why the ndarray object itself isn't tracked though? I
> >> must say I find this puzzling; the only thing I can think of is that
> >> the python compiler notices that data isn't used anymore after its
> >> creation, and deletes it right after its creation as an optimization,
> >> but that conflicts with my own experience of the GC.
> >>
> >>
> >
> > Not sure if it does, but my quick try and error says:
> > In [15]: class T(tuple):
> >     ....:     pass
> >     ....:
> >
> > In [16]: t = T()
> >
> > In [17]: objs = [o for o in gc.get_objects() if isinstance(o, T)]
> >
> > In [18]: objs
> > Out[18]: [()]
> >
> > In [19]: a = 123.
> >
> > In [20]: objs = [o for o in gc.get_objects() if isinstance(o, float)]
> >
> > In [21]: objs
> > Out[21]: []
> >
> > So I guess nothing is tracked, unless it contains things, and numpy
> > arrays don't say they can contain things (i.e. no traverse).
> >
> > - Sebastian
> >
> >
> >
> >> _______________________________________________
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
>
> --
> +---------------------------------------------------------+
> | Mads Ipsen                                              |
> +----------------------+----------------------------------+
> | Gåsebæksvej 7, 4. tv | phone:              +45-29716388 |
> | DK-2500 Valby        | email:      mads.ipsen at gmail.com |
> | Denmark              | map  :   www.tinyurl.com/ns52fpa |
> +----------------------+----------------------------------+
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140915/41574c3c/attachment.html>