[Import-sig] Re: Proposal for a modified import mechanism.

[Gordon McMillan]

> Hey Gordon,
> 
> > eric wrote:
> >
> > [Frederic Giacometti]
> >
> > > > because the obvious reason of increased name space
> > > > collision, increased run-time overhead etc...
> > >
> > > I'm missing something here because I don't understand why
> > > this increases name space collision.
> >
> > Currently, os.py in a package masks the real one from
> > anywhere inside the package. This would extend that to
> > anywhere inside any nested subpackage. Whether that's a
> > "neat" or a "dirty" trick is pretty subjective. The wider the
> > namespace you can trample on, the more it tends to be "dirty".
> 
> Yeah, I guess I come down on the "neat" side in this one.  If I
> have a module or package called 'common' at the top level of a
> deep hierarchy, I'd like all sub-packages to inherit it.  

I have no objection to having an easy way of getting to 
common from subpackages.

[snip]

> Also, if someone has the gall to hijack os.py at the top of your
> package directory structure, it seems very likely you want this
> new behavior everywhere within your package.

Perhaps. The trouble is that under this proposal there is no 
way to get to the real os.py, even when you know what's going 
on. The proposal doesn't fix a flaw in today's import, it extends 
the flaw. If there were different ways of spelling "absolute" and 
"relative", you would get what you want, and I would get what I 
want.
 
> I have a feeling this discussion has been around the block a few
> times when packages were first being developed...

Not really. I think it was Mark Hammond that convinced Guido 
of the usefulness of packages. It got hacked into import.c. It 
worked for the small existing population of packages and it 
basically hasn't been touched since then.

Package authors certainly haven't ended up following Java's 
model.
 
> >
> > > If the objection is to the fact
> > > that SciPy can have a version of Numeric in it that masks a
> > > Numeric installed in site-packages, I guess I consider this a
> > > feature, not a bug.  Afterall, this is already the behavior
> > > for single level packages, extending it to multi-level
> > > packages seems natural.  If this isn't your objection, please
> > > explain.
> >
> > Well, it's a feature that can crash Python. If the package
> > (which the user has, and you have a hijacked, incompatible copy
> > of) contains an extension module, all kinds of nasty things can
> > happen when both are loaded.
> 
> This I need to know about.  How does this happen?  So you have
> two extension modules, with the same name, one living in a
> package and the other living in site-packages.  If you import
> both of these, their namespaces don't conflict in some strange
> way do they?
> 
> Or are you talking about passing a structure (like a numeric
> array) generated in one ext module into a routine in the other
> ext module (expecting a different format of a numeric array) and
> then getting some strange (seg-fault even) kind of behavior? 
> Anyway, I'd like a few more details for reasons orthogonal to
> this discussion.

Much depends on platform. Windows will happily load 2, so 
errors are likely to be of the 2nd sort (although it may not take 
explicitly passing something around - the extension may be 
relying on TLS data, for example).

On platforms that do a real dynamic link, it's going to be C 
symbols that get stomped on.

[snip]

> I haven't followed the discussion as to whether the standard
> library will move to packages, but I'll be surpised if it does. 

It's been brought up (and delayed) with each Python release 
since packages became official.

> I very much like the encapsulation offered by packages, but have
> found their current incarnation difficult to develop compared to
> simple modules. The current discussion concerns only one of the
> issues.

And it's my contention that the problem this proposal "fixes" is 
a symptom of a deeper problem - the fact that the import 
namespace has a concept of "relative" but no way to spell it. 
The proposal makes some things work, and makes others 
worse.
 
> As for overhead, I thought I'd get a couple more data points from
> distutils and xml since they are standard packages.  The
> distutils import is pretty much a 0% hit.  However, the xml
> import is *much* slower -- a factor of 3.5.  

Prabhu objects to your numbers. Perhaps he's using the 
standard xml package, and you're using _xmlplus? The latter 
is vastly more complex.

- Gordon