[Python-ideas] PEP 511: Add a check function to decide if a "language extension" code transformer should be used or not

[Brett Cannon]

On Wed, 27 Jan 2016 at 13:34 Andrew Barnert <abarnert at yahoo.com> wrote:

> On Jan 27, 2016, at 12:20, Brett Cannon <brett at python.org> wrote:
>
> On Wed, 27 Jan 2016 at 08:49 Andrew Barnert via Python-ideas <
> python-ideas at python.org> wrote:
>
>
>> If someone builds a transformer that only runs on files with a different
>> extension, he already needs an import hook, so he might as well just call
>> his transformer from the input hook, same as he does today.
>>
>
> And the import hook is not that difficult.
>
>
> Unless it has to work in 2.7 and 3.3 (or, worse, 2.6 and 3.2). :)
>

Sure, but you're already asking for a lot of pain if you're trying to be
that compatible at the AST/bytecode level so I view this as the least of
your worries. :)


>
> You can reuse everything from importlib without modification except for
> needing to override a single method in some loader to do your
> transformation
>
>
> Yes, as of 3.4, the design is amazing. In fact, hooking any level--lookup,
> source, AST, bytecode, or pyc--is about as easy as it could be.
>
> My only complaint is that it's not easy enough to find out how easy import
> hooks are. When I tell people "you could write a simple import hook to play
> with that idea", they get a look of fear and panic that's completely
> unwarranted and just drop their cool idea. (I wonder if having complete
> examples of a simple global-transformer hook and a simple special-extension
> hook at the start of the docs would be enough to solve that problem?)
>

So two things. One is that there is an Examples section in the importlib
docs for 3.6: https://docs.python.org/3.6/library/importlib.html#examples .
As of right now it only covers use-cases that the `imp` module provided
since that's the most common thing I get asked about.

Second, while it's much easier than it has ever been to do fancy stuff with
import, it's a balancing act of promoting it and discouraging it. :) Mess
up your import and it can be rather hard to debug. And this is especially
true if you hook in early enough such that you start to screw up stuff in
the stdlib and not just your own code. It can also lead to people going a
bit overboard with things (hence why I kept my life simple with the
LazyLoader and actively discourage its use unless you're sure you need it).
So it's a balance of "look at this shiny thing!" and "be careful because
you might come out screaming".


>
> And I'm a bit worried that if Victor tries to make things like MacroPy and
> Hy easier, it still won't be enough for real-life cases, so all it'll do is
> discourage people from going right to writing import hooks and seeing how
> easy that already is.
>

We don't need to empower every use-case as much as possible. While we're
consenting adults, we also try to prevent people from making their own
lives harder. All of this stuff is a tough balancing act to get right.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160128/84dec855/attachment-0001.html>