[C++-sig] Re: Refactoring, Compilation Speed, Pyste, Lua/Ruby/JavaScript...

[David Abrahams]

Please know that I take my own response with a grain of salt; if
potential contributors really think they want the presentation you
describe, who am I to argue?

"Brett Calcott" <brett.calcott at paradise.net.nz> writes:

>> Well, I'd be happy to collaborate with you on an implementation
>> document.  It's not something I can afford to do by myself, in part
>> because I don't know what questions to answer.  If you would agree
>> to have a conversation with me about it here, and from that create
>> a RestructuredText document which we'll include in the CVS, I'd be
>> more than happy to type out details at length.  I think you might
>> have to dig into the code a little, too.  Deal?
>
> Deal. It be a bit slow at first as I am relocating to another country in the
> next 10 days (NZ -> Aussie)

Interesting change.  How come?

> so my internet connection will be sporadic, and I'll be busy
> finding a new home and settling in.

OK, I'm patient.

> I'd like to approach it as though we are trying to write a simplified
> version of boost.python, ignoring compiler workarounds, and not using the
> preproc. We'll show a coded python C extension, an equivalent in
> boost.python syntax, then go through the structures needed for the
> boost.python code to generate the equivalently operating C extension code
> that we initially showed.

I think that sounds like it will cover a lot of details that nobody
needs to see, and in particular there are many things supported by
Boost.Python (derived<->base conversions, overloading, even simple
stuff like classes and static method support) that we don't know how
to do in 'C' without replicating large swaths of Boost.Python
functionality so the document would end up being as much about 'C'
extension writing as about Boost.Python internals.

I'm hoping that this document will be useful to other people who want
to be involved with the project (and *me* as well), and I worry that
by starting from the top level, what you're describing will fail to
illuminate the architecture of Boost.Python, which IMO is the real
obstacle to understanding the implementation.

> We could work through examples like the ones on this page :
>
> http://starship.python.net/crew/arcege/extwriting/pyext.html
>
>
> ie.
>
> python extension:
> =================
> PyObject *python_add(self, args)....
> ...
> ...
> ..
>
>
> PyMethodDef methods[] = {
> {
>     "add", MyCommand, METH_VARARGS},
>     {NULL, NULL},
> };
>
>
> void initexample()
> {
>     (void)Py_InitModule("add", methods);
> }
>
>
> equivalent in boost
> ==================
> ...
>     def("add", python_add);
>
>
> Explain how the boost example generated equivalent code to above using very
> cut-down techniques that capture the 'essence' of the boost.python
> implementation, but not the detail.

a. I think you need to understand much of the detail.

b. The code generated by Boost.Python isn't equivalent to the above in
   any trivial sense. Almost everything in traditional 'C' extension
   building is geared towards initializing arrays of (function)
   pointers and constants which the Python core treats almost as a
   program, interpreting elements of the array and generating Python
   objects (e.g. callables) from them.  Boost.Python doesn't/can't
   take advantage of these mechanisms, so it builds the Python objects
   directly.  I guess what I'm saying is that the mapping is not very
   direct so I don't see a comparison with the 'C' API extension is
   going to help.


> =================
> 1. The module def
> 2. how is the methods array created (or simulated)
> 3. How does argument parsing happen..
> ...etc..
>
>
> Does this make sense?
> Comments (from anyone)?

If you are convinced this is the right way to go, I'm still happy to
help and answer questions.  I would prefer to do a bottom-up
description, starting with the core and working outward... especially
because it would help the luabind people who are interested in
integration with Boost.Python.  A top-down description will always be
describing new things in terms of their components which are concepts
nobody understands yet.  I'm not going to argue too strongly with
anyone who's volunteering, though!

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com