[Import-SIG] Proposed design for importlib.resources()

[Brett Cannon]

On Tue, 24 Nov 2015 at 14:13 Barry Warsaw <barry at python.org> wrote:

> On Nov 24, 2015, at 08:21 PM, Brett Cannon wrote:
>
> >> Module API vs package API.  Doesn't pkg_resources actually support
> >> something similar to both, with the module function providing a
> convenience
> >> API?
> >
> >Yes, but that doesn't sway me. This isn't a "pkg_resources++" but a "make
> >reading data from a package make sense in a modern import world". IOW I'm
> >purposefully not using pkg_resources as a template but simply as a
> >motivating factor.
>
> You're not taking into account a migration path for existing users of
> pkg_resources.  If you make it difficult to convert, then people are much
> less
> likely to do it for existing code, despite the ability to remove a
> dependency.
>

It's one of those situations where it's balancing future code with
migrating old code. I'm doing this to solve the problem of importlib
lacking any standardized way to get at resources in a package, not to
migrate pkg_resources users who want to eliminate that dependency (although
that would be a perk).


>
> If my existing code already has
>
>     from pkg_resources import resource_string as resource_bytes
>
> all I'd need to do is change this one line to
>
>     from importlib.resource import read_bytes as resource_bytes
>
> and I'm done.  If I need to support multiple versions of Python, I can even
> do:
>
>     try:
>         from importlib.resource import read_bytes as resource_bytes
>     except ImportError:
>         from pkg_resources import resource_string as resource_bytes
>
> Without this API, it's much more difficult for me to convert my existing
> code,
> either incrementally or whole-hog, because now I have to either add that
> convenience function myself (and import it everywhere) or rewrite all my
> call
> sites.  Why bother?
>

Nothing is stopping people from writing their own pkg_resources
compatibility layer. Hell, I will promise to create shim_resources or
something and put it on PyPI for those that want a really simple migration
path. But the base API that goes into the stdlib and will need to be
supported forever doesn't need to go the way of compatibility if its going
to feel out of place in importlib (which the pkg_resources API will,
especially if we put a method on loaders to get a resource loader).


>
> >Unfortunately for you the poll liked the other approach and TOOWTDI. So
> >either convince me that resources.read_bytes(pkg, path) is better than
> >resources(pkg).read_bytes(path) or consider the bike shed painted. :)
>
> It's not better or worse, it's just different.  As pkg_resources has
> shown, it
> doesn't have to be either-or.
>
> I never saw the poll since I don't pay attention to Google+.


Which is why I also linked to it on Twitter. :) I actually tried to do it
on twitter initially but guess whose poll support restricts option lengths
so much you can't type a method call out. :p



  How
representative are those 59 votes of the current pkg_resource users and
potential future users of this API?  If I had seen the poll I would have
complained that it didn't give me a chance to choose both APIs <wink>.

>I'm not convinced it's necessary to provide an equivalent open() yet;

Right, I'm not necessarily advocating for it, just describing what it would
have to do if it were there.  It's something I occasionally wish I had, but
all the building blocks are there to invent it when needed.

>> I do have one use of resource_listdir() which is used to find importable
>> plugin modules at runtime.  It's handy.
>
>I'm going to punt on this for as long as possible because it's asking for
>trouble to get right. For example, if I do resources(pkg).listdir(), then I
>will end up returning relative paths, but if you disassociate those paths
>from pkg then you have lost proper context. You could return tuples of
>(pkg, relative_path), but that just doesn't seem satisfactory either. I'm
>just not convinced yet it is needed enough to support (at least initially).

It's a tougher API to recreate from the building blocks, so it would be nice
not to have to reinvent the wheel everywhere, but it's also a much less
common
API.  I'm not at all worried about the disassociation problem, since
os.listdir() gives you relative paths anyway so it's a familiar behavior.

Yeah, I realize it's something you can't make from scratch, but I'm still
going to avoid it while I can because as soon as this goes in then people
are going to want a similar API for discovering modules in a package and
would abuse this API if they don't get the other API.

-brett

Cheers,
-Barry
_______________________________________________
Import-SIG mailing list
Import-SIG at python.org
https <https://mail.python.org/mailman/listinfo/import-sig>://
<https://mail.python.org/mailman/listinfo/import-sig>mail.python.org
<https://mail.python.org/mailman/listinfo/import-sig>/mailman/
<https://mail.python.org/mailman/listinfo/import-sig>listinfo
<https://mail.python.org/mailman/listinfo/import-sig>/
<https://mail.python.org/mailman/listinfo/import-sig>import-sig
<https://mail.python.org/mailman/listinfo/import-sig>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/import-sig/attachments/20151124/7bbf0ae6/attachment.html>