[Python-3000] packages in the stdlib

[Jim Jewett]

On 6/2/06, Talin <talin at acm.org> wrote:

> ... it seems to me that there is a cross-linkage
> between the "reorganize standard library" task and the "refactor import
> machinery" task

Eventually, yes.  As Brett pointed out, "reorganize the standard
library" stands on its own, and is intended to make finding modules
easier.

The tasks get linked when the library again grows, or when 3rd-party
packages try to replace (or superset) the functionality.  Then we
might start caring that package X is exactly the stdlib package X
(which was sufficient and tested against), or that it be Xplus (which
the sysadmin or user says is a faster and bugfixed superset).

>- in that much of the arguments about the stdlib names
> seem to hinge on policy decisions as to (a) whether 3rd party libs
> should be allowed to co-mingle with the stdlib modules,

By default, yes, but it should be easy to tell which you have if you do care.
So (pretending that wx is in the stdlib, because it has a short name)

    import UI.wx     # Import a module claiming to implement the wx interface

    import py.UI.wx    # Import exactly the wx that was installed with
the standard lib.


> and (b) what
> kinds of co-mingling should be allowed ('monkeypatching', et al), and
> (c) what specific import mechanisms should these 3rd-party modules have
> access to in order to do this co-mingling.

These are not related to the stdlib reorg.  The only catch is that
with a deeper namespace, some 3rd party packages will know where they
belong, and it makes sense to let them say so.  Namespace packages
(let alone tags) are not in the standard library now, so this can't be
done as cleanly.

> Moreover, past threads on the topic of import machinery have given me
> the vague sense that there is a lot of accumulated cruft in the way that
> packages are
...
> I say 'vague sense' because even after reading all these threads, I only
> have a murky idea of what actual *problems* all of these various
> improvements are trying to solve.

Those that I'm vaguely aware of:

(1)  It is hard to split packages.  The idiom of module.py importing
_module sort of works for a two-way split of a single module, but
splitting modules and subpackages across different locations doesn't
work so nicely.  Putting .pyc in one place and .py in another is a
recurring minor itch.

(2)  Every import extension reinvents the wheel, and only one wheel at
a time.  Whether the file is in a zip archive or not should be
unrelated to whether it is a .pyo file or a cheetah template --- but
currently isn't.

(3)  As these one-off wheels build up, it becomes difficult to know
where something really came from (and how), so it is harder to find
the "real" package and easier to test (or ship) the wrong version.

-jJ