[C++-sig] multiply defined to_python_converter

Langston, Matthew David langston at SLAC.Stanford.EDU
Wed Dec 11 20:00:18 CET 2002

If every Boost.Python module had their own registry (as opposed to a
global registry), wouldn't this solve the problem? It would just mean
that type conversions for any type other than the Python built-ins
might be duplicated from time to time.

But this might not be a bad thing, since there are potentially an
infinite number of ways to convert these more complex types. For
example, I thought it was obvious that a std::vector<int> should
just convert to a Python tuple. It never even occurred to me that
someone might actually want a class_<std::vector<> > instead.

My current problem with the global registry was simply solved by
commenting out my C++ to_python_converter and recompiling since, in
this particular instance, I could rely on the third-party library to
supply the conversion functionality that I needed. However, this
forces an implicit dependency of my module on the third-party module.
There is no way for me to distribute my module standalone.

The "explicit is better than implicit" argument would seem to apply
here. It would be more work for the Boost.Python module writer, but
they would have full control over their conversions and their modules
wouldn't have any implicit dependencies on other modules.


-----Original Message-----
From: David Abrahams [mailto:dave at boost-consulting.com] 
Sent: Wednesday, December 11, 2002 9:34 AM
To: c++-sig at python.org
Subject: Re: [C++-sig] multiply defined to_python_converter

"Ralf W. Grosse-Kunstleve" <rwgk at yahoo.com> writes:

> --- David Abrahams <dave at boost-consulting.com> wrote:
>> > How do I work around this problem of "multiply defined 
>> > to_python_converter"?
>> Hmm.  I'm not sure what the best answer is.  Does Python have a 
>> warning mechanism?
> http://www.python.org/doc/current/lib/module-warnings.html
>> Maybe it would be best to maks such multiple
>> definitions trigger a Python warning instead of a hard error.
> I am not sure if this is a viable solution. For example there are two 
> plausible alternatives for std::vector<> to_python conversions:
> - to a Python list or tuple
> - implied by class_<std::vector<> >
> I am using both alternatives, depending on the value type. Others will 
> have different requirements and make different choices. This will lead 
> to incompatibilities.
> Ways out?
> Boost.Python could simply occupy the most prominent ecological niches 
> by default, then everybody has to adhere to that. Conflicts will be 
> the exception. Unfortunately a fully fledged std::vector<> wrapper is 
> fairly heavy-weight. But there is a similar wrapper in the scitbx 
> (flex_wrapper.h; see Boost.Python FAQ) if someone wants to take on 
> this chore.
> Another idea would be a conversion registry with "context" or 
> "namespaces". Any function could be defined in a certain context with 
> associated conversions. But I am afraid David will need a BIG grant to 
> implement such a beast.

Let's think about what the design should be before we worry about actually getting it done. ;-)

In Boost.Python v1, all conversions were "aggressively local".  The only way to export them was through a relatively cumbersome explcit construct in both the exporting and importing modules (No offense, Ralf)!  One thing I worry about a little is that we might have gone too far in the direction of automatically publishing all conversions for v2:

  1. Boost.Python extensions (and the Boost.Python library) compiled
  with different compilers *cannot* be used together unless the
  compilers' ABIs match in many respects:

        type_info (or on some platforms, type_info::name) must have
        the same format.

        Object and vtable layout of any types used across modules must
        match exactly.

  2. Since shipping Boost.Python v2, we've discovered problems with
     some EDG front-ends (used in lots of compilers) on those
     platforms which use type_info::name comparison, which even
     prevent modules compiled with the same compiler from
     interoperating properly unless a cumbersome hack is used.  The
     problem is that the compiler will choose different
     representations of a given type_info::name() depending on how the
     type is instantiated.

It's not clear to me how important the ABI issue is, but I can imagine that it might be quite important.  For example, most Windows compilers are capable of generating compatible object and vtable layouts in the single-inheritance case (which covers an important subset of all C++
applications) even though they may have incompatible type_info formats. 

I'm not really aware of how similar or different different compiler ABIs are on other platforms, though I think the new ABI developed by a consortium for IA64 may mean we're evolving towards a common ABI for many platforms -- making the issue of incompatibility moot.  However, we're surely not there yet.

Another approach we could have taken for conversions would be to have all conversions registered in a "local registry" with each module, and use an explicit import. Something like:

    // Just sketching, here:
    import<Y> import_Y("Foo.Bar.Y");      // Absolute package path
    import<X> import_X("../Foo/Bar.X");   // Relative package path

I realize this is in some sense a step backwards, but I think it's important to convince ourselves we really have the right solution by weighing the pros and cons.

                       David Abrahams
   dave at boost-consulting.com * http://www.boost-consulting.com Boost support, enhancements, training, and commercial distribution

C++-sig mailing list
C++-sig at python.org

More information about the Cplusplus-sig mailing list