[Python-Dev] Does Zip Importer have to be Special?
phil at riverbankcomputing.com
Thu Jul 24 20:12:13 CEST 2014
On 24/07/2014 6:48 pm, Brett Cannon wrote:
> On Thu Jul 24 2014 at 1:07:12 PM, Phil Thompson
> <phil at riverbankcomputing.com>
>> I have an importer for use in applications that embed an interpreter
>> that does a similar job to the Zip importer (except that the storage
>> a C data structure rather than a .zip file). Just like the Zip
>> I need to import my importer and add it to sys.path_hooks. However the
>> earliest opportunity I have to do this is after the Py_Initialize()
>> returns - but this is too late because some parts of the standard
>> library have already needed to be imported.
>> My current workaround is to include a modified version of
>> as a frozen module that has the necessary steps added to the end of
>> _install() function.
>> The Zip importer doesn't have this problem because it gets special
>> treatment - the call to its equivalent code is hard-coded and happens
>> exactly when needed.
>> What would help is a table of functions that were called where
>> _PyImportZip_Init() is currently called. By default the only entry in
>> the table would be _PyImportZip_Init. There would be a way of
>> the table, either like how PyImport_FrozenModules is handled or how
>> Inittab is handled.
>> ...or if there is a better solution that I have missed that doesn't
>> require a modified _bootstrap.py.
> Basically you want a way to specify arguments into
> importlib._bootstrap._install() so that sys.path_hooks and
> were configurable instead of hard-coded (it could also be done just
> importlib being installed, but that's a minor detail). Either way there
> technically no reason not to allow for it, just lack of motivation
> this would only come up for people who embed the interpreter AND have a
> custom importer which affects loading the stdlib as well (any reason
> can't freeze the stdblib as a solution?).
Not really. I'd lose the compression my importer implements.
(Are there any problems with freezing packages rather than simple
> We could go the route of some static array that people could modify.
> Another option would be to allow for the specification of a single
> which is called just prior to importing the rest of the stdlib,
> The problem with all of this is you are essentially asking for a hook
> let you have code have access to the interpreter state before it is
> initialized. Zipimport and the various bits of code that get loaded
> startup are special since they are coded to avoid touching anything
> isn't ready to be used. So if we expose something that allows access
> to full initialization it would have to be documented as having no
> guarantees of interpreter state, etc. so we are not held to some API
> makes future improvements difficult.
> IOW allowing for easy patching of Python is probably the best option I
> think of. Would tweaking importlib._bootstrap._install() to accept
> specified values for sys.meta_path and sys.path_hooks be enough so that
> can change the call site for those functions?
My importer runs under PathFinder so it needs sys.path as well (and
doesn't need sys.meta_path).
More information about the Python-Dev