[IronPython] object lifecycle issues

[William Reade]

Hi Dino

I don't *think* that's the case -- the stub has a __del__ method, that 
should never get called, purely to work around that bug (or possibly a 
subtly different one: I internally characterised it as "an object won't 
get __del__ed unless the type it was created as had a __del__ method, at 
the time it was created", which doesn't *quite* match your example). I 
guess it's one of the ones that never made it onto Codeplex... it's 
there now, at 
http://ironpython.codeplex.com/WorkItem/View.aspx?WorkItemId=23564

The repro for for issue #2 -- which requires current ironclad with 
appropriate logging -- is attached as x.py. The output, which you can 
see a bit further down, shows what happens when you 
construct-then-delete instances of various numpy data types. In short, 
the int8 and float32 dtypes don't include a CLR type among their 
immediate bases, and do get deleted correctly, while the int32 and 
float64 dtypes do and don't respectively.

Anyway, I'll look into your earlier suggestions -- thank you very much!

x.py output (after a lot of noise from the numpy import)
------------------------------------------------------------
========================================

start <class 'numpy.int8'>

real new1 <class 'numpy.int8'>
fake new1 _ironclad_class_stub (<class 'numpy.signedinteger'>,)
fake new2 (object) 2107
fake init <class 'unknown._ironclad_class_stub'> 2107
real new2 <class 'numpy.int8'> 2107

constructed; id is 2107

real del 2107

finished <class 'numpy.int8'>

========================================

start <class 'numpy.int32'>

real new1 <class 'numpy.int32'>
fake new1 _ironclad_class_stub (<class 'numpy.signedinteger'>, <type 'int'>)
fake new2 (int) 2111
fake init <class 'unknown._ironclad_class_stub'> 2111
real new2 <class 'numpy.int32'> 2111

constructed; id is 2111


finished <class 'numpy.int32'>

========================================

start <class 'numpy.float32'>

real new1 <class 'numpy.float32'>
fake new1 _ironclad_class_stub (<class 'numpy.floating'>,)
fake new2 (object) 2116
fake init <class 'unknown._ironclad_class_stub'> 2116
real new2 <class 'numpy.float32'> 2116

constructed; id is 2116

real del 2116

finished <class 'numpy.float32'>

========================================

start <class 'numpy.float64'>

real new1 <class 'numpy.float64'>
fake new1 _ironclad_class_stub (<class 'numpy.floating'>, <type 'float'>)
fake new2 (float) 2121
fake init <class 'unknown._ironclad_class_stub'> 2121
real new2 <class 'numpy.float64'> 2121

constructed; id is 2121


finished <class 'numpy.float64'>
------------------------------------------------------------

...and the internal logging is as follows:

------------------------------------------------------------
        public const string CLASS_STUB_CODE = @"
def __new__(cls, *args, **kwargs):
    print 'fake new1', cls.__name__, cls.__bases__
    if issubclass(cls, int):
        result = int.__new__(cls, args[0])
        print 'fake new2 (int)', id(result)
        return result
    if issubclass(cls, float):
        result = float.__new__(cls, args[0])
        print 'fake new2 (float)', id(result)
        return result
    result = object.__new__(cls)
    print 'fake new2 (object)', id(result)
    return result

def __init__(self, *args, **kwargs):
    print 'fake init', type(self), id(self)

def __del__(self):
    print 'fake del', id(self)

def __setattr__(self, name, value):
    object.__setattr__(self, name, value)

_ironclad_class_stub = _ironclad_metaclass('_ironclad_class_stub', 
_ironclad_bases, {
    '__new__': __new__,
    '__init__': __init__,
    '__del__': __del__,
    '__setattr__': __setattr__,
})
";

        public const string CLASS_CODE = @"
def __new__(cls, *args, **kwargs):
    print 'real new1', cls
    result = cls._dispatcher.newfunc('{0}.tp_new', cls, args, kwargs)
    print 'real new2', cls, id(result)
    return result

def __del__(self):
    print 'real del', id(self)
    self._dispatcher.ic_destroy(self)

_ironclad_class_attrs['__new__'] = __new__
_ironclad_class_attrs['__del__'] = __del__

_ironclad_class = _ironclad_metaclass('{0}', _ironclad_bases, 
_ironclad_class_attrs)
_ironclad_class.__doc__ = '''{2}'''
_ironclad_class.__module__ = '{1}'
";
------------------------------------------------------------

...while the code that actually constructs ipy objects around cpy 
objects looks like this:

------------------------------------------------------------
       
        private void
        ActualiseArbitraryObject(IntPtr ptr)
        {
            IntPtr typePtr = CPyMarshal.ReadPtrField(ptr, 
typeof(PyObject), "ob_type");
            PythonType type_ = (PythonType)this.Retrieve(typePtr);

            object[] args = new object[]{};
            if (Builtin.issubclass(type_, TypeCache.Int32))
            {
                args = new object[] { CPyMarshal.ReadIntField(ptr, 
typeof(PyIntObject), "ob_ival") };
            }
            if (Builtin.issubclass(type_, TypeCache.Double))
            {
                args = new object[] { CPyMarshal.ReadDoubleField(ptr, 
typeof(PyFloatObject), "ob_fval") };
            }
            // ...
           
            object obj = PythonCalls.Call(this.classStubs[typePtr], args);
            Builtin.setattr(this.scratchContext, obj, "__class__", type_);
            // ...
        }
------------------------------------------------------------

BTW: As it happens, the stub class is now a sibling of the real class 
rather than a subclass, because it feels cleaner. Regardless, the 
behaviour is identical in each case.

Cheers
william

Dino Viehland wrote:
> Could this be issue 2?
>
> class Real(object):
>     def __new__(cls, *args, **kwargs):
>         print 'real new'
>         return object.__new__(Stub)
>     #def __del__(self): pass     # uncomment me and this works as expected
>
> class Stub(Real):
>     def __del__(self):
>         print "I've been finalized"
>
> f = Real(1.0)
> del f
>
> import sys
> if sys.platform == 'clr':
>     import clr
>     from System import GC
>     for _ in range(4):
>         GC.Collect()
>         GC.WaitForPendingFinalizers()
>
>
>   
>> -----Original Message-----
>> From: users-bounces at lists.ironpython.com [mailto:users-
>> bounces at lists.ironpython.com] On Behalf Of William Reade
>> Sent: Monday, July 20, 2009 9:38 AM
>> To: Discussion of IronPython
>> Subject: [IronPython] object lifecycle issues
>>
>> Hi all
>>
>> I have two problems that are at least somewhat related:
>>
>> +++ Issue 1 (probably your bug):
>>
>> ---------------------------------------------------------------
>> C:\dev\ironclad - Copy>ipy y.py
>> real new
>> stub new
>> real init
>> real del
>>
>> C:\dev\ironclad - Copy>python y.py
>> real new
>> stub new
>> stub init
>> real del
>> ---------------------------------------------------------------
>>
>> I freely admit that the attached code is pretty weird, but I really do
>> need to do stuff like this in Ironclad. The difference in behaviour may
>> or may not be responsible for certain failing numpy/scipy tests -- I'm
>> not sure how to tell -- but I'd enormously appreciate a fix.
>>
>> I'd report the issue on Codeplex, but trying to visit the issue tracker
>> just leaves my browser spinning forever. Speaking of which: is there
>> any alternative way of reporting bugs that doesn't make me feel as if
>> I'm spamming the list with out-of-band noise? I'm pretty sure that a
>> few bugs have just dropped off my stack in the last few months, just
>> because I got tired of waiting for Codeplex to start working.
>>
>> +++ Issue 2 (almost certainly my bug):
>>
>> In a nearly identical* situation -- close enough that I can't say how
>> it's actually different -- f will never get its __del__ method called
>> (the object is destroyed -- a WeakReference to it knows it's dead --
>> but the __del__ call never happens).
>>
>> For context: I have *very* similar classes, whose instances are
>> constructed in exactly the weird way demonstrated in the attached file,
>> and which work fine. The only difference between the cases that work
>> and the cases that don't is that the broken cases multiply inherit from
>> ipy types which wrap CLR types (int and float (and maybe str, although
>> I haven't tested that one)), while the working cases have simple chains
>> of single inheritance from user-defined types all the way up to object.
>> However, the attached repro doesn't show my problem, so it's clearly
>> not
>> *just* to do with multiply inheriting from CLR types.
>>
>> Does anyone have any idea what I might be doing wrong? I know this is a
>> vague request, but I'm running out of ideas, and I'd really appreciate
>> some input from someone who understands precisely what all those ipy
>> MetaFoo classes are doing.
>>
>> Cheers
>> william
>>
>>
>> * the attached file started life as an attempt to repro the __del__
>> issue, and I incidentally noticed the __init__ issue.
>>     
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>   

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: x.py
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20090721/2ebc1cb5/attachment.ksh>