[Python-ideas] Refactoring the import system to be object-oriented at the top level

[Brett Cannon]

On Sun, Nov 8, 2009 at 13:20, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Brett Cannon wrote:
>>> The underlying import machinery would migrate to instance methods of the
>>> new class.
>>>
>>
>> Do you really mean methods or just instance attributes? I personally
>> don't care personally, but it does require more of an API design
>> otherwise.
>
> I did mean methods, but I also realise how much work would be involved
> in actually following up on this idea. (As the saying goes, real
> innovation is 1% inspiration, 99% perspiration!)
>
> If you don't move the machinery itself into instance methods then you
> just end up having to pass the storage object around to various
> functions. Might as well make that parameter 'self' and use methods.
>

I don't quite follow. What difference does it make if they are
instance attributes compared to methods? The data still needs to be
stored somewhere that is unique per instance to get the semantics you
want.

The other thing you could do with this is provide import_module() on
the object so it is a fully self-contained object that can do an
entire import on its own without having to touch anything else (heck,
you could even go so far as to have their own module cache, but that
might be too far as all loaders currently are expected to work with
sys.modules).

>>> The standard import engine instance would be stored in a new sys module
>>> attribute (e.g. 'sys.import_engine'). For backwards compatibility, the
>>> existing sys attributes would remain as references to the relevant
>>> instance attributes of the standard engine.
>>
>> How would that work? Because they are module attributes there is no
>> way to use a property to have them return what the current
>> sys.import_engine uses.
>
> Yes, I eventually realised it would be better to turn the dependency
> around the other way (i.e. have an engine subclass that used properties
> to refer to the sys module attributes)

Yeah, you could have it default to the attributes on the sys module if
no instance attributes are set.

>
>>> Modules would get a new special attribute (e.g. '__import_engine__')
>>> identifying the import engine that was used to import that module.
>>> __import__ would be modified to take the new special attribute into account.
>>>
>>
>> Take into account how? As in when importing a package to always use
>> the import engine used for the parent module in the package?
>
> Yes, that's what I was thinking. That would be necessary to allow
> operations like the runpy methods to execute without having side effects
> on the main import machinery the way they do now.
>
> Having run_path and run_module functions that were as side-effect free
> (and hence thread-safe) as exec and execfile would be kind of cool.
>

That would be nice to have.

>>> I suspect such a change would greatly simplify experimentation with
>>> Python security models as well: restricted code could be given a
>>> restricted import engine rather than the restrictions having to be baked
>>> in to the standard import engine.
>>>
>>
>> Huh, I wonder made you think about that example? =)
>
> Not *just* your efforts over the last few years, although those were
> definitely a major inspiration :)
>
>>> >From an OO design point of view, it's a classic migration of global
>>> state and multiple functions to manipulate that state into a single
>>> global instance of a new class.
>>
>> If anything it makes it easier to discover everything that affects
>> importing instead of having to crawl through the sys docs to find
>> every attribute that happens to mention the word "import".
>
> Heck, *I* had to stare at dir(sys) for a while to make the list of
> import-related attributes in my post and I've been working on import
> related code for years. Even then I almost missed 'dont_write_bytecode'
> and wouldn't be the least surprised if someone pointed out that I
> actually had missed something else :)

Yeah, there are a lot of them.