[Import-SIG] Proposal for a lazy-loading finder

[Nick Coghlan]

On 11 July 2017 at 04:48, Brett Cannon <brett at python.org> wrote:
>
>
> On Sun, 9 Jul 2017 at 06:59 Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> On 9 July 2017 at 20:03, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> > I would suggest allow people to fully customize the blacklist /
>> > whitelist logic using a callable (because looking up by fullname is a
>> > bit inflexible).
>>
>> Oh, I like that - and then we'd have "ensure_lazy" and "ensure_eager"
>> as callback factories that accepted a predefined list of names.
>
>
> If we provide an optional callable argument then I would drop the
> whitelist/ensure_lazy option. It's easier to explain and the common case
> will be blacklisting a module for lazy loading if you're implicitly flipping
> it on. For the whitelist case we can add importlib.util.lazy_import() and
> people can just be explicit (if this is important enough to even care
> about).

Sorry, I was overly terse. By callback factories, I meant something like:

    def ensure_eager_exact(names):
        """Disable lazy loading for named modules"""
        names = set(names)
        def lazy_load_filter(fullname):
            return fullname not in names
        return lazy_load_filter

    def ensure_lazy_exact(names):
        """Only enable lazy loading for named modules"""
        names = set(names)
        def lazy_load_filter(fullname):
            return fullname in names
        return lazy_load_filter

    def ensure_eager_by_prefix(names):
        """Disable lazy loading for named modules and their submodules"""
        names = set(names)
        def lazy_load_filter(fullname):
            parts = fullname.split(".")
            prefixes = (".".join(parts[:i]) for i in range(len(parts)))
            return all(prefix not in names for prefix in prefixes)
        return lazy_load_filter

    def ensure_lazy_by_prefix(names):
        """Only enable lazy loading for named modules and their submodules"""
        names = set(names)
        def lazy_load_filter(fullname):
            parts = fullname.split(".")
            prefixes = (".".join(parts[:i]) for i in range(len(parts)))
            return any(prefix in names for prefix in prefixes)
        return lazy_load_filter

Those could even just be recipes in the documentation rather than
actual standard library functions.

> And I purposefully didn't do a prefix match for ensure_eager as it's only
> meant for specific modules that have some try/except block which fails in
> the face of lazy loading. And since that should be a per-module thing
> instead of a per-package thing I don't want it over-extending. Plus
> providing a callback solution lets people engineer their own prefix matching
> solution if that's what they need.

Yep, that's why I like Antoine's callback suggestion.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia