[Import-SIG] Proposed design for importlib.resources()

[Brett Cannon]

If we make it e.g.,  __loader__.resources().read_bytes(path) then I may be
more amenable to creating a importlib.resources module with the bastardized
pkg_resources API. Going to have to think about it, though.

On Tue, 24 Nov 2015, 16:25 Brett Cannon <brett at python.org> wrote:

> On Tue, 24 Nov 2015 at 14:13 Barry Warsaw <barry at python.org> wrote:
>
>> On Nov 24, 2015, at 08:21 PM, Brett Cannon wrote:
>>
>> >> Module API vs package API.  Doesn't pkg_resources actually support
>> >> something similar to both, with the module function providing a
>> convenience
>> >> API?
>> >
>> >Yes, but that doesn't sway me. This isn't a "pkg_resources++" but a "make
>> >reading data from a package make sense in a modern import world". IOW I'm
>> >purposefully not using pkg_resources as a template but simply as a
>> >motivating factor.
>>
>> You're not taking into account a migration path for existing users of
>> pkg_resources.  If you make it difficult to convert, then people are much
>> less
>> likely to do it for existing code, despite the ability to remove a
>> dependency.
>>
>
> It's one of those situations where it's balancing future code with
> migrating old code. I'm doing this to solve the problem of importlib
> lacking any standardized way to get at resources in a package, not to
> migrate pkg_resources users who want to eliminate that dependency (although
> that would be a perk).
>
>
>>
>> If my existing code already has
>>
>>     from pkg_resources import resource_string as resource_bytes
>>
>> all I'd need to do is change this one line to
>>
>>     from importlib.resource import read_bytes as resource_bytes
>>
>> and I'm done.  If I need to support multiple versions of Python, I can
>> even
>> do:
>>
>>     try:
>>         from importlib.resource import read_bytes as resource_bytes
>>     except ImportError:
>>         from pkg_resources import resource_string as resource_bytes
>>
>> Without this API, it's much more difficult for me to convert my existing
>> code,
>> either incrementally or whole-hog, because now I have to either add that
>> convenience function myself (and import it everywhere) or rewrite all my
>> call
>> sites.  Why bother?
>>
>
> Nothing is stopping people from writing their own pkg_resources
> compatibility layer. Hell, I will promise to create shim_resources or
> something and put it on PyPI for those that want a really simple migration
> path. But the base API that goes into the stdlib and will need to be
> supported forever doesn't need to go the way of compatibility if its going
> to feel out of place in importlib (which the pkg_resources API will,
> especially if we put a method on loaders to get a resource loader).
>
>
>>
>> >Unfortunately for you the poll liked the other approach and TOOWTDI. So
>> >either convince me that resources.read_bytes(pkg, path) is better than
>> >resources(pkg).read_bytes(path) or consider the bike shed painted. :)
>>
>> It's not better or worse, it's just different.  As pkg_resources has
>> shown, it
>> doesn't have to be either-or.
>>
>> I never saw the poll since I don't pay attention to Google+.
>
>
> Which is why I also linked to it on Twitter. :) I actually tried to do it
> on twitter initially but guess whose poll support restricts option lengths
> so much you can't type a method call out. :p
>
>
>
>   How
> representative are those 59 votes of the current pkg_resource users and
> potential future users of this API?  If I had seen the poll I would have
> complained that it didn't give me a chance to choose both APIs <wink>.
>
> >I'm not convinced it's necessary to provide an equivalent open() yet;
>
> Right, I'm not necessarily advocating for it, just describing what it would
> have to do if it were there.  It's something I occasionally wish I had, but
> all the building blocks are there to invent it when needed.
>
> >> I do have one use of resource_listdir() which is used to find importable
> >> plugin modules at runtime.  It's handy.
> >
> >I'm going to punt on this for as long as possible because it's asking for
> >trouble to get right. For example, if I do resources(pkg).listdir(), then
> I
> >will end up returning relative paths, but if you disassociate those paths
> >from pkg then you have lost proper context. You could return tuples of
> >(pkg, relative_path), but that just doesn't seem satisfactory either. I'm
> >just not convinced yet it is needed enough to support (at least
> initially).
>
> It's a tougher API to recreate from the building blocks, so it would be
> nice
> not to have to reinvent the wheel everywhere, but it's also a much less
> common
> API.  I'm not at all worried about the disassociation problem, since
> os.listdir() gives you relative paths anyway so it's a familiar behavior.
>
> Yeah, I realize it's something you can't make from scratch, but I'm still
> going to avoid it while I can because as soon as this goes in then people
> are going to want a similar API for discovering modules in a package and
> would abuse this API if they don't get the other API.
>
> -brett
>
> Cheers,
> -Barry
> _______________________________________________
> Import-SIG mailing list
> Import-SIG at python.org
> https <https://mail.python.org/mailman/listinfo/import-sig>://
> <https://mail.python.org/mailman/listinfo/import-sig>mail.python.org
> <https://mail.python.org/mailman/listinfo/import-sig>/mailman/
> <https://mail.python.org/mailman/listinfo/import-sig>listinfo
> <https://mail.python.org/mailman/listinfo/import-sig>/
> <https://mail.python.org/mailman/listinfo/import-sig>import-sig
> <https://mail.python.org/mailman/listinfo/import-sig>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/import-sig/attachments/20151125/25fc63c9/attachment-0001.html>