[Python-ideas] Packages and Import

[Brett Cannon]

On 2/9/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> "Brett Cannon" <brett at python.org> wrote:
> > On 2/9/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> [snip]
>
> > > Now, I tried the first of those lines in Python 2.5 and I was surpised
> > > that having two files foo and goo, goo importing foo via the first
> > > example above, didn't work.  What is even worse is that over a year ago
> > > I was working on an import semantic for relative imports that would have
> > > made the above do as I would have expected.
> > >
> > > This leads me to believe that *something* about relative imports is
> > > broken, but being that I mostly skipped the earlier portions of this
> > > particular thread, I'm not certain what it is.  I would *guess* that it
> > > has to do with the way the current importer comes up with package
> > > relative imports, and I believe it could be fixed by switching to a
> > > path-relative import.
> > >
> > > That is, when module goo is performing 'from . import foo', you don't
> > > look at goo.__name__ to determine where to look for foo, you look at
> > > goo.__file__ .
> >
> > But what about modules stored in a sqlite3 database?  How is that
> > supposed to work?  What made basing relative imports off of __name__
> > so nice is that it allowed the import machinery figure out what the
> > resulting absolute module name was.  That allowed the modules to be
> > stored any way that you wanted without making any assumptions about
> > how the modules are stored or their location is determined by an
> > importer's find_module method.
>
> How is it done now?

Importers and loaders; see PEP 302.

>  Presumably if you have some module imported from a
> database (who does this, really?),

No one at the moment.  But think of it as an alternative to zip files
if you use sqlite3 for the DB storage.  Several people have told me
privately they would like to see an importer and loader for this.

> it gets a name like
> dbimported.name1.name2, which an import hook can recognize as being
> imported from a database.  Now, is dbimported.name1.name2 really the
> content of an __init__.py file (if it was a package), or is it a module
> in dbimported.name1?
>
>
> Right now, we can't distinguish (based on __name__) between the cases of...
>
>     foo/
>         __init__.py
>
> and
>     foo.py
>

Right, but you can based on whether __path__ is defined, which is how
import tells.

> But we can, trivially, distinguish just by *examining* __file__ (at
> least for files from a filesystem). For example:
>
>     >>> import bar
>     >>> import baz
>     >>> bar.__name__, bar.__file__
>     ('bar', 'bar.py')
>     >>> baz.__name__, baz.__file__
>     ('baz', 'baz\\__init__.py')
>     >>>
>
> It's pretty obvious to me which one is a package and which one is a
> module.
>

Right, but as I said, this is what __path__ is for; to tell when a
module is just a module or a package.  And then the __name__ attribute
is used to tell if it is a top-level module or not.

> If we codify the requirement that __file__ must end with '.../__init__.X'
> (where X can be; py, pyw, pyc, so, dll, pyd, etc.) if the thing we
> imported is a package, then the import hooks don't need to use the
> __file__ attribute for anything other than discerning between "is this a
> package, or is this a module", and can then handle the __name__ mangling
> as per import semantics.  The only trick is if someone were to
> specifically import an __init__ module (from .package import __init__),
> but even then, the results are garbage (you get the module's __init__
> method).
>
> > But it isn't a file path, it's an absolute module name that you are after.
>
> If one were to just do "path" manipulations, afterwards you can
> translate that to an absolute name (perhaps based on the path of
> __main__). Pretending that you have a path can make the semantic
> non-ambigous, but I prefer the alternate I just described.
>
>
> > > It also naturally leads to a __name__ semantic that Guido had suggested
> > > to me when I was talking about relative imports:
> > >
> > >     goo.__name__ == '__main__'
> > >     foo.__name__ == '__main__.foo'
> > >     baz.__name__ == '__main__..bar.baz'
> > >
> > > Which could more or less be used with the current importer; it just
> > > needs a special-casing of 'from . import ...' in the __main__ module.
> >
> > And I am trying to avoid special-casing for this.
>
> And it's only really an issue because we can't currently discern between
> module or package, right?

Basically.  It's also where within a package the module is, so it
isn't quite that simple.

The issue comes down to that we lose where a module is contained
within a package hierarchy once __name__ gets set to __main__.  If we
came up with another way to delineate that a module was being executed
or some other way to specify where a module was in a package then the
problem would be solved.  I prefer trying to change the former since
the latter is perfectly handled with __name__ as-is when it isn't
changed to __main__.

-Brett