[Python-Dev] _PyImport_LoadDynamicModule questions

[Guido van Rossum]

> > > However, I still think that people creating --enable-shared
> > > installations are misguided: You gain nothing (IMO), and you lose a
> > > number of benefits:
> > 
> > Do you understand why people have always been asking for this?  Are
> > they all misguided?  It really is a FAQ (3.30).  Why?
> 
> People give various reasons:
> 
> - (from #400938): "Using a shared library should have an advantage if
>   you're running multiple instances of Python (be it standalone
>   interpreter or embedded applications)."
>   This is nonsense, of course: the interpreter executable is shared
>   just as well.

It's not nonsense if you have lots of *different* programs that embed
the interpreter.  If each has a static copy, those static copies
aren't shared between different copies.

> - libraries should be shared (405931). There is often no further
>   rationale given, but I believe "... because that saves disk space"
>   is the common implication. Given that /usr/local/bin/python would be
>   the only application of libpythonxy.so on most installation, this
>   rationale is questionable.

Agreed.  Disk space is cheap.

> - it simplifies embedding (with the variant "embedding is not possible
>   without it"). Some people are simply not aware that a libpython.a is
>   also installed. In 497102, James Henstridge argues that PyXPCOM
>   mandates a shared libpython, as does gnome-vfs. He might have a case
>   here, but I think a special-case shared library that exposes all
>   Python symbols might be more appropriate.

Possibly, but this could be countered with "if we're going to have to
provide a shared library anyway, it might as well be the only one we
offer."  I don't understand why PyXPCOM needs a shared library, but it
may point to something we'll hear more in the future -- shared
libraries are more open to inspection.

I also wonder if the ability to slip a (compatible) newer version of a
shared library in might not make a good argument for using shared
libraries.  Use case: you have an app that embeds Python.  The
embedded Python version uses the same standard library location as the
installed Python, say, Python 2.1.2.  Now you upgrade to Python 2.1.3.
The standard library is upgraded.  (a) Wouldn't it be nice if the
embedding app was automatically upgraded too (rather than having to
relink it, which may be a pain if the source and objects were thrown
away).  (b) Some new feature might be added to the core (in a
binary compatible way) that is used by some library module.  If the
embedding app uses that module, it will fail because the statically
linked app still is Python 2.1.2.

> - on the same topic, the PostgreSQL documentation claims that you
>   cannot build PL/Python without a shared libpython. They admit that
>   it might work to use the static library, and that it is just the
>   fault of their build system to not support this scenario:
> http://www.postgresql.org/idocs/index.php?plpython-install.html

I expect we'll see this more and more.  After all, every library that
comes with Linux is a shared lib (even / especially libc).

> - For embedded applications, people also bring up "allows sharing at
>   run-time" argument). In that case, it is factually true. However,
>   even without a shared libpython, multiple copies of the same
>   embedded executable (or shared library) will still share code.

See above.

> - The Windows port uses a python DLL.
> 
> To summarize, it probably does have advantages for people who want to
> embed Python in applications that are themselves shared libraries. I
> think those advantages are outweighed by the problems people may get
> with a shared libpython, and who never want to embed Python.

What problems (apart from the pain of getting this to build right on
many platforms)?  You mentioned a bit of a slowdown due to PIF code
(probably not even measurable in pystone) and a slower startup due to
a few stat calls.  Note that all extensions are already shared
libraries -- so the problems can't be too bad. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)