Konrad Hinsen writes:
> > I would much more prefere have Python try to import from the
> > paltform independant part of a package (installed under $prefix)
> > and, if it cannot find what we are looking for, lookup
> > "automatically" the platform-dependant part of that package.
> Shouldn't that be the other way round? I'd expect to be able to override
> general modules by platform-specific modules.
Greg Ward and I were talking about this stuff the other day, and I
think we decided that there was no good way to have multiple
implementations of a module installed such that the platform dependent
version was sure to take precedence over a platform independent
version; this relies on the sequence of directories in the relevant
search path (whether it be sys.path or a package's __path__).
The general solution seems to be that two things need to be done: a
package's __path__ needs to include *all* the appropriate directories
found along the search path, not just the one holding __init__.py*,
AND the platform dependent modules should have different names from
the platform independent modules. The platform independent module
should be the public interface, and it can load platform dependent
code the same way that string loads functions from strop.
The problem here is that the package's __path__ is not being created
this way now; if anyone has time to work on a patch for
Python/import.c, I'd be glad to help test it! ;-)
Fred L. Drake, Jr. <fdrake(a)acm.org>
Corporation for National Research Initiatives
[Problem with dynamic extensions in packages being platform dependent]
I haven't followed the thread too closely, but was alarmed by
the recent proposals of splitting .so files out of the "normal"
package distribution under a separate dir tree. This is really
not such a good idea because it would cause the package information
stored in the extension module to be lost (you can't have two
top-level packages with the same name on the path: only the first one
on the path will be used).
Here is the scheme I would use: create a subpackage for the
extension and have it take care of importing the correct
shared lib for the platform Python is currently running on.
The libs themselves could be placed in plat-<platform> subdirs
of that subpackage and the __init__.py would then load the
shared lib using either a sys.path+__import__() hack or
thread safe via imp.load_dynamic().
An even simpler solution is installing the whole package under
.../python1.5/plat-<platform> separately for each supported
platform rather than putting it under site-packages. [Disk space
is no argument nowadays and its likely that different platforms
need different Setup files anyway.]
Y2000: 224 days left
Python Pages: http://www.lemburg.com/python/
(I just subscribed here, so maybe I've missed earlier replies to David's
post in the PSA list)
At 11:57 AM -0700 5/20/99, David Ascher wrote:
>I'm (slowly) getting to the point where I agree. Two thoughts:
>1) imputil.py (greg stein's thing) might be a good place to start working
> out a better system. See distutils-sig for URLs.
>2) the problem of statically compiled package-enclosed modules is
> separate and needs to be addressed in the core. In other words, it
> won't make it before 1.6.
Point 2 shouldn't be too hard. It is already possible (since a 1.5.2 alpha
I think) to statically link submodules in a frozen build. I guess it's
relatively easy to patch find_module() to do something like this:
foo.bar is registered as a "builtin" in config.c file as
(Hm, this is problemetic if the is a distinct global builtin module "bar")
find_module() should then first check sys.builtin_module_names with the
full name before doing anything else. (probably only when it is confirmed
that "foo" is a package.)
No time to play with that right now, but it sure seems trivial.
[I'm going to try to yank this thread over from PSA-members,
which should have been done *long* ago!]
[on psa-members, Michel Sanner opined:]
> I do not think that trying to bend over bacakwards to circumevent this
> "limitation" is the right way to proceed .. unless changing the import
> mechanism in Python itself is something that would be extremely difficult to
I agree that a change to the import mechanism is in order. My
understanding (from another one of those office-hallway conversations
with Fred) is that it would be very sensible to add a package's
platform-dependent directory to the package's __path__ attribute. No
frobbing of sys.path is necessary, and thus no danger of stupid name
conflicts in extension modules that are supposed to be buried deep in
some package structure.
Perhaps imputil.py can help us in the playing around stage; I'm not
familiar with it, though, so I'll refrain from further comment.
Greg Ward - software developer gward(a)cnri.reston.va.us
Corporation for National Research Initiatives
1895 Preston White Drive voice: +1-703-620-8990
Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913
It seems to me that the platform dependent and independent trees of a Python
installation are not symetric in some sense:
on one side we have:
on the other side
what bothers me is that we do not have that extra level under lib in the
platform independent tree. I'd like to have something like:
$prefix/lib/python$version/standard/ (equivalent of lib-dynload)
$prefix/lib/python$version/packages/ (for paltform independant packages)
and these directories should be part of the Python PATH built by default.
I am not sure where pur Python packages are supposed to be installed right now
I am once again in the Python update phase and (of course) running into the
same type of problems regarding the installation of packages.
I believe that Python should provide support for installing packages that
contain both platform independant files (.py) and platform dependant files
(.so, .dso .pyd). The reason is that I do not want to install (and maintain)
multiple copies of the .py (one for each paltform I support). It seems to me
that this is one ogf the benefits of platform independance. The problem right
now is that the only way to do this (I know of) if to hack together an
__init__.py file for the package placed in the paltform independant part of the
installation tree and that would add the right directory to the PATH for
importing .so files.
The problem with that is that I need to do that for every single package.
Would it be unreasonable to have the Python import mechanism check for packages
in the $prefix AND the $exec_prefix directory ?