[C++-sig] Re: C++/Boost vs. Python object memory footprint

[David Abrahams]

Stephen Davies <chalky at ieee.org> writes:

> On Thu, 2002-12-12 at 00:22, David Abrahams wrote:
>> Stephen,
>> 
>> Was my answer helpful?
>
> Kind of. You gave lots of useful info, but no clear answer (to my maybe
> not so clear) questions: will the memory usage decrease? and will the
> speed increase? I guess there is no clear answer, since it depends on
> how it's used.

Right.

>> David Abrahams <dave at boost-consulting.com> writes:
>> 
>> > Stephen Davies <chalky at ieee.org> writes:
>> >
>> >> I had 350MB of AST in memory yesterday and my system didn't cope too
>> >> well. I figure each object has at least one dictionary which can't be
>> >> cheap memory wise. The question is, will I still have proxy objects
>> >> floating around if I use Boost.Python, or will the python code directly
>> >> use the C++ objects without creating the instance dictionary et.al.?
>> >
>> > All the code for implementing C++ object wrappers is in
>> > libs/python/src/object/class.cpp.  Instance dictionaries are created
>> > only "on demand", the first time the instance's __dict__ attribute is
>> > accessed (see instance_get_dict), but I have no idea whether that
>> > tends to happen almost always or almost never.
>
> Hmm, it would be useful to investigate what causes this, eg: all
> accesses to member variables/calls, getattr calls, or just
> "foo.__dict__" statements.
>
> Hmm I've just been reading through Python sources and boost's class.cpp
> for the last hour, 

Ouch, that's the hard way.  Probably much easier to use a debugger.

> and here's what I've got:
>
> Python/ceval.c: Attribute access of an object uses PyObject_GetAttr. 
>
> Objects/Object.c: GetAttr uses the tp_getattro/tp_getattr methods in the
> Type object. Boost does not set these, so the default is used.
> PyObject_GenericGetAttr or type_getattro (not sure which) uses some kind
> of Descriptor Object, and if that fails, the dictionary. The descriptor
> stuff confused me, but class.cpp has 0 in the slots for
> tp_descr_get/set.

That doesn't mean anything. This instance type is derived from
Python's "object" type, so lots of slots get filled in automatically
by the system during PyType_Ready.  No, the internals of Python's new
type system are not really documented anywhere :(

> So, it seems that the dict would be used after all, for every method
> call.

I wouldn't count on it.  Remember also that the class' dict and the
instance dict are two separate beasts.

FWIW, I have had it in the back of my mind to allow classes to be
wrapped with no instance dict at all for quite a while, but it hasn't
been a priority.

> I'm not sure how I got to 350MB, but I suspect that the dict is a
> large part of it, since there are a lot of (simple: get/set and
> visitor accept) methods, of which only the accept (and destructor of
> course) would be virtual in C++. I don't know either how memory
> efficient the Python data structures are (PyList, PyTuple, even just
> pointers), but there's a large chance that there is a *lot* of small
> chunks of memory being allocated - iirc each chunk has some
> overhead. I'm sure C++ could store this more efficiently.

Probably.  In Pure Python, tuples are better than lists, though.  If
you have your choice, use tuples.

> If you're interested in looking at the AST and Type heirarchies, there
> is both a Python version in Synopsis/Core/{AST,Type}.py and a mostly
> corresponding C++ version in Synopsis/Parser/C++/syn/{ast,type}.hh.

Maybe after I dig myself out of this backlog... remind me again if you
feel it would be helpful, and I'll try.

>> > In general, a wrapped C++ object with a corresponding Python object is
>> > the size of a new-style class (derived from 'object' in Python)
>> > instance plus the extra size required to allow variable-length data in
>> > the instance, plus the size of the C++ object, plus the size of a
>> > vtable pointer, plus a pointer to the C++ object's instanceholder,
>> > plus zero or more bytes of padding required to ensure that the
>> > instanceholder is properly aligned.  
>
> This sounds like a lot, but without the dict of every method and
> variable it may still be smaller.

I think you're confusing classes with instances.  Methods get stored
once per class, so they're relatively cheap, and wrapped Boost.Python
methods take up about as much as Python methods do.  Attributes get
stored once per instance, and are relatively expensive.  Boost.Python
exposes C++ data members using "property" descriptors, so there's the
cost of one or two methods in the class, plus just the raw storage for
the corresponding C++ object in the instance.  Usually, this will cost
less than a Python attribute in an instance dictionary.

>> > I'm not sure what you mean by your question "will I still have proxy
>> > objects floating around...?" 
>
> Some API wrappers create proxy objects with methods corresponding to the
> API like:
> class GTK_Widget (something):
> 	def get_child(self):
> 		return _gtkmodule.do_get_child(self._obj)
>
> I've since realised that Boost doesn't resort to such trickery.

Nope.  The new type system makes all that unneccessary.

>> > If your C++ data structure contains pointers or smart pointers, you
>> > can arrange for Python objects to be created which only embed those
>> > pointers (instance<pointer_holder<Ptr> >). These Python objects will
>> > be in existence only as long as your Python code holds a reference to
>> > them. So, for example, it should be possible for Python code to do a
>> > walk over your C++ AST, with only O(log(N)) Python objects in
>> > existence corresponding to those N C++ objects at any given time.
>
> This sounds good, though I wonder at the overhead of creating and
> destroying those pointer_holder instances - the AST can contain 10,000's
> of objects, and be walked several times over. 

It's a trade-off between memory and speed.  But if your memory image
is pushing things out of the cache, sometimes it's not a trade-off
anymore ;-)

> Would this work if the nodes had something like
> "std::vector<AST::Declaration*> declarations" ?

Your question's not very precise, but if I understand it, yes.
However, for an application like this I think I'd recommend
std::vector<boost::shared_ptr<AST::Declaration> >

>> >> A related question is what the speed is like calling the C++ objects vs.
>> >> normal Python objects. I can't imagine it would be slower. 
>> >
>> > I haven't done any tests, but it certainly could be slower if used
>> > poorly.  There is some overhead at the Python/C++ boundary associated
>> > with looking for eligible type converters, overloading, etc.  However,
>> > I imagine that ends up being negligible in most cases.  The best use
>> > of a Boost.Python C++ binding puts a large amount of computation on
>> > the C++ side of the language boundary.  One way I could imagine
>> > slowing down a Python program would be to translate a very large
>> > number of trivial functions to C++.  C++ function wrappers will
>> > certainly occupy more memory than the corresponding Python code would,
>> > and this could eventually affect cache locality.
>
> More memory for every object instance? 

No, more memory in the executable code image.

> The methods in the AST are indeed very simple with the exception of
> the constructors which do type checking, and the AST class which has
> some other stuff.
>
> It sounds like there wouldn't be much overhead at the boundary:
> since the AST was first written in Python, there is no method
> overloading.

I think you're basically right, but the lack of overloading in the
original design doesn't have much impact on it.

>> >> Perhaps you can add these to the Boost.Python FAQ :)
>> >
>> > If you'll help me edit them into a suitable form, I'd be most happy
>> > to!
>
> Sure, but I'm not happy with the answers yet. 

Let's keep going until you are.

> It looks like I'm going to have to actually come up with some test
> cases and see what happens. I don't know when I'll have time for
> that though.

I'll run a quick test now to see when __dict__ gets filled in.

-- 
                       David Abrahams
   dave at boost-consulting.com * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution