[C++-sig] Patches and complete pyste replacement prototype for pyplusplus

[Roman Yakovenko]

Sorry I push wrong button.
> On 2/27/06, Allen Bierbaum <abierbaum at gmail.com> wrote:
> > > Allen has a class called "Module" that is used to control the internals
> > > of pyplusplus. In my version, this class is called "Pipeline" (as in my
> > > opinion, this class actually represents the pyplusplus "core" I find the
> > > name "Module" a bit misleading. But some might say this is nitpicking... ;)
> >
> > I debated for quite a while what to call this object.  It corresponds
> > roughly to a builder for what pyplusplus calls a module, so I went
> > with that.  Another reason I chose this name is that from the user
> > perspective what they are trying to build is a python module and this
> > is the tool they are using to build it.  I am definitely willing to
> > rename this and thinking about it now it may be better to call it
> > "ModuleBuilder" or something along those lines.
>
I would like to call it generator_t, but module_builder_t is also okay.

> > In Allen's version, the user always explicitly creates an instance of
> that class himself, in my version this instance is created internally
> > and each method is also available as function which internally calls the
> > corresponding method of the global instance (if desired the user could
> > also create an instance himself).
>
> This is definitely one area where our efforts diverged.  I really took
> hold of the idea early on to use an object oriented API throughout
> because:
> - I am assuming that people using the tool know python
> - I want the ability to have binding generation scripts instantiate
> multiple separate builders
> - It seemed to be a good idea conceptually to deal with objects throughout
>
> I could definitely add a similar interface of global methods that
> automatically call through to a single global instance, but it
> wouldn't work for my bindings so I didn't spend to much time on it.
> If this is a required capability I could easily add it.

I think object oriented interface is fine. I will not let global variables to
come into pyplusplus without good reason. More over, I think, that within same
script it should be possible to use more then one module_builder_t( I
like this name ).

> > In Allen's version there are three main "control methods": parse(),
> > createCreators(), createModule(). In my version, I have the three
> > methods parse(), codeCreators() and writeFiles() which serve the same
> > purpose (as said above, these methods are also available as functions).
> > In both versions, the second step (creating the code creators) is done
> > internally if it wasn't done explicitly by the user (in my version I
> > also applied that rule for the parse() step, but I admit that probably
> > everyone has to do that step manually anyway (but it's a nice feature
> > for a "Hello World" example :) ).

I can think about an other approach: properties
For example, class module_builder_t will have 4 properties:
- parser configuration
   keeps all data to configure pygccxml parser
- code creators factory configuration
   keeps all data to configure module_creator.creator_t class
- declarations
   returns declarations
   within this property, files will be parsed, only once, and
declaration tree will be returned.
- module_creator
   returns module_t code creator that has been created using by creator_t class

In this way user don't need to think "parsing" and "code creators
factory", but rather
I have a set of declarations, lets do some adaptation.


> > In both versions, there are methods Class, Method, Function, etc. to
> > select one or more particular declarations that can then be decorated to
> > customize the final bindings. In Allen's version, these function either
> > return a DeclWrapper or MultiDeclWrapper object (depending on whether
> > the selection contains one or more declarations). In my version, the
> > return value is an IDecl object (that always acts like a MultiDeclWrapper).
> > Decorating the declarations also looks almost the same in both versions.
>
> I thought about doing this similar to Matthias, but I decided that I
> wanted an easy ability to detect user errors and give good warnings.
> What I found was that by splitting this is two I could have a separate
> interface for MultiDeclWrapper (the case where multiple declarations
> are wrapped) and only allow methods that made sense for multiple
> declarations.  Similarly this interface can modify the way the methods
> operate to make them take into account they they are wrapping multiple
> declarations.   If I made everything wrap multiple declarations then I
> would have to add test/handling code in each method to check wether
> the method was valid.

I am second to that opinion. Also I think it is very easy to make
Matthias happy.

mb = module_builder_t( ... )
mb.class_ = mb.class_group

You replace function that return 1 class with function that returns
many classes.
Your code will work without changes.

Also we can not join between decl_wrapper and multi_decl_wrapper.
Every declaration
has set of unique properties like parent or location. Those properties
will not be in interface
of multi_decl_wrapper.

> I am not too hung up on this though as it was more an implementation
> detail then anything else.
>
> > So far, both versions are almost identical. However, at the moment, a
> > big difference is the expressiveness of the declaration selection
> > methods (Class, Method, Function,...) and the exact semantics of the
> > declaration wrappers. And here I actually prefer my version where
> > obtaining an IDecl object is like doing a database query to retrieve a
> > set of declarations that meet certain requirements. The resulting IDecl
> > object can reference an arbitrary number of declarations scattered all
> > over the declaration tree. It can even be empty and still provide the
> > decoration interface. Such a "database query" can be further refined by
> > calling the Class, Method, Function,... methods again on a IDecl object
> > (this is a feature that the MultiDeclWrapper in Allen's version
> > currently does not allow). Each individual query can also be composed of
> > several filters where different filter types are concatenated with AND
> > and filters of the same type are concatenated with OR (see
> > http://i31www.ira.uka.de/~baas/pypp/classpyppapi_1_1decl_1_1_i_decl.html#063f1880bb6bc164d2f0ecd5fc92a3c1).
> > In Allen's version, all queries are based on the declaration type and
> > name. Queries for methods can optionally use the arguments or return
> > type as well (but the name is still mandatory). As this is a subset of
> > the query filters in my version I think it wouldn't be a problem to
> > elevate Allen's version to the same expressiveness as my version.

Yes, we need both approaches. Using declaration type, like variable_t, it
is very easy and intuitive. But some time it should be possible to say something
like this: give me all declarations that their names start with QXml or QDom.

> There is one area here though where I am a little worried.  Namely I
> find the way I query only the children of a declaration to be a little
> more structured.
>
> For example with my method the user would always go about build up
> their module based on the name hierarchy of the module:
>
> ns = mod.Namespace("test_ns")
> class1 = ns.Class("class1")
> class1_method1 = class1.Method("method1")
> class2 = ns.Class("class2")
> class2_method1 = class2.Method("method1")
>
> In Matthias's API I believe you could do something where you could ask
> for all methods named "method1" across the entire decl tree.  I am not
> sure this is such a  good idea or at least I would classify it as a
> power user capability.  As such I am not opposed at all to adding it
> but I don't think it should be the default and I would recommend that
> it be available through a different interface.

Well, I think that for the first version we will implement Matthias's API.
Using it, is very easy to mimic what you want:

mb = module_builder_t(...)

class1 = mb.class( name="class1" )
class1_method1 = mb.member_function( name=class1.decl_string + "method1" )

I agree, that this is a little bit verbose, but this should be okay
for the first version.
In future we definitely will have your way within the code. It has an
other advance: speed.

> Agreed, it could be very useful to have a way to make explicit that a
> regex is being used.

I like Allen's implementation. You have one class that all it's job is to match
name or something else to regular expression. Then you can use
function composition
to create a filter you need.

> Do you think it would be a good idea for me to refine my prototype
> with your query system or should we start over with a new code base
> merging the best ideas?

Yes, but wait a little. I made some changes in pygccxml and pyplusplus
to make it easier
to introduce those changes.

> -Allen

Thank you

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/