[Distutils] Accessing package data files in wheels

[Paul Moore]

On 29 June 2015 at 10:26, Paul Sokolovsky <pmiscml at gmail.com> wrote:
> and yet we're stuck at the old base PEPs which
> overlooked providing stream access protocol for package resources
> access.

The PEP did not "overlook" stream access. Rather, the compatibility
constraints and the need to support existing code meant that we needed
to ensure that we required the minimal possible interface from
loaders. Even get_data was an optional interface.

In practice, many of the constraints around at the time no longer
apply, and zip and filesystem loaders remain the most common examples,
so the conservative approach of PEP 302 can be revisited (as I said).
But someone needs to step up and manage such a change before it will
happen.

> Let that be rhetoric question then, and let everyone assume that so
> far trading eggs for wheels was more important than closing a visible
> accidental gap in the stdlib/loader API.

The egg->wheel transition was about *distribution* formats. The loader
API is a runtime facility. The two are unrelated.

One of the problems with eggs was the fact that they were a combined
installation and runtime format, so confusing the two aspects is
understandable (but still incorrect).

> There was recent discussion on python-dev how other languages are
> cooler because they allow to create self-contained executables. Python
> has all parts of the equation already - e.g. the standard way to put
> Python source inside an executable is using frozen modules. So, that's
> another usecase for accessing package resources - was support for
> frozen modules was implemented at all?

See https://docs.python.org/3.5/library/importlib.html#importlib.machinery.FrozenImporter

>From that, it appears that the frozen module importer does not
implement the ResourceLoader API. So no, get_data is not supported for
frozen modules. Of course, you can write your own extension of
FrozenImporter for your application, so it's entirely possible to get
this to work. But the standard Python bootstrap process (which is
FrozenImporter's main use case, AFAIK) doesn't need that feature,
which is probably why it's not present.

Anyway, as you can see, all the various mechanisms are available, and
extending importlib is certainly possible, so as we've said it's
really only about someone with the motivation doing the work. It could
probably even be done as a 3rd party project in the first place (much
like pkg_resources was) and then proposed for inclusion in the stdlib
once it has been found to be useful.

Paul