[Python-3000] Should package __init__ files include pkgutil.extend_path?
Phillip J. Eby
pje at telecommunity.com
Tue Sep 9 22:31:22 CEST 2008
At 12:38 PM 9/9/2008 -0700, Brett Cannon wrote:
>On Tue, Sep 9, 2008 at 10:06 AM, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 03:28 PM 9/6/2008 -0700, Brett Cannon wrote:
> >> On Sat, Sep 6, 2008 at 2:06 PM, <skip at pobox.com> wrote:
> >> > I'm trying to figure out how to install this dbm.sqlite module I have
> >> > without overwriting the basic install. My thought was to create a dbm
> >> > package in site-packages then copy sqlite.py there. That doesn't work
> >> > though. Modifying dbm.__init__.py to include this does:
> >> >
> >> > import pkgutil
> >> > __path__ = pkgutil.extend_path(__path__, __name__)
> >> >
> >> > I'm wondering if all the core packages in 3.x should include the above
> >> > in
> >> > their __init__.py files.
> >> >
> >> Well, a side-effect of this is that all package imports will suddenly
> >> spike the number of stat calls linearly to the number of entries on
> >> sys.path.
> > "All package imports"? "Spike"?
>pkgutil.extend_path() would be executed for every package imported by
>the fact that is code at the global level of the module.
Each package that uses it and that is imported, yes.
> And if you
>look at the implementation of extend_path(), there is a
>os.path.isdir() call for every entry on sys.path, and if that succeeds
>there is os.path.isfile() call. Plus there is also an os.path.isfile()
>call for every sys.path entry as well.
Note, btw, that that could be greatly reduced by use of
sys.path_importer_cache; only entries that are missing or None need
to have the subdirectory check.
>I call that a "spike" in "all package imports" in terms of stat calls
>if this was added to all packages as suggested. And that can be
>painful on systems where stat calls are expensive (e.g., NFS). At
>least extend_path() appends so the new entries are put at the back of
...which actually negates the entire point of the proposal, which was
somebody wanting to be able to install an override/upgrade of a
module in a stdlib package.
>Yes, it's a trade-off depending on what penalty cost you would prefer
>to pay. But as I said, I don't like the idea of letting people inject
>into the stdlib namespace like this in the first place so I don't want
>this to happen in any official capacity.
IIUC, the OP was requesting the ability to *upgrade* a
stdlib-provided module, not add items to the namespace.
More information about the Python-3000