[Python-Dev] Integrating Expat

[Greg Stein]

On Mon, Oct 08, 2001 at 10:03:54AM +0200, Martin v. Loewis wrote:
> > I think bundling the sources is fine, but it should *ONLY* be a fallback
> > if you do not find the Expat library installed on the system. *ALWAYS*
> > link against a system-installed library first.
> 
> I'm not so sure about that. There are many problems doing that:
> 
> - if there is no shared libexpat, a static one is only usable if it is
>   compiled either as PIC, or if pyexpat is statically linked into
>   the Python interpreter. These conditions are impossible to test.

What? I've never seen a .a installed on a system that isn't usable. What
makes Python modules so special that they could not use a library? I find
your statement rather surprising, and I'm quite doubtful of the issue.

I've linked .a files into .so modules often. Never a problem.

> - if the installed version of expat is older than the one we ship,
>   I think Python should still use the included one.

Every time you use the included one, you run into the possibility of
symbol conflict problems. If you refuse to use the one installed in the
system, then you increase the chance of problems.

Granted: there isn't much you can do if pyexpat *requires* something newer
than what is on the system (does it require the latest? or can pyexpat work
with all versions of expat?).

> I think the problem you had with Apache does not apply to Python:
> 
> > Apache 1.3 would get loaded and export the Expat symbols to the rest
> > of the process.
> 
> In Python, symbols of one extension module are not exported to
> anything else.

Eh? Again: what makes Python so special? How can you guarantee this rule? On
*all* platforms?

> We would not build a libexpat.so (not sure whether
> Apache did),

We never built a .so. The library was linked right into the Apache httpd
executable.

> and nobody would link with pyexpat.so. Furthermore,
> dynloading pyexpat.so will use RTLD_LOCAL, unless that is overridden
> by the application (and you really have to know what you're doing when
> changing the dlopen flags).

This argument doesn't hold. dlopen() is not used on every platform. Thus,
you cannot specify RTLD_LOCAL on all platforms, thus you cannot guarantee
that symbols in a library will remain private.

> > Now, some *other* module is loaded and dynamically links against
> > /usr/lib/libexpat.so. Now you have *two* sets of Expat symbols and
> > crashes are going to start happening.
> 
> No, that can't happen. pyexpat.so does not export its symbols (due to
> RTLD_LOCAL), so the other module will still use libexpat.so.

See above.


Incorporating Expat directly into Python will definitely be "nice", but I've
been through this already. It isn't pretty. If that shared library exists,
then use the darned thing, or you're gonna get burned.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/