[Python-Dev] Solving the import-deadlock case

[Nick Coghlan]

On 3 Jul 2013 04:34, "Pascal Chambon" <pythoniks at gmail.com> wrote:
>
> Hello everyone,
>
> I'd like to bring your attention to this issue, since it touches the
fundamentals of python's import workflow:
> http://bugs.python.org/issue17716
>
> I've tried to post it on the python-import ML for weeks, but it must
still be blocked somewhere in a moderation queue, so here I come ^^
>
> TLDR version: because of the way import current works, if importing a
package "temporarily" fails whereas importing one of its children
succeeded, we reach an unusable state, all subsequent attempts at importing
that package will fail if a "from...import" is used somewhere. Typically,
it makes a web worker broken, even though the typical behaviour of such
process woudl be to retry loading, again and again, the failing view.
>
> I agree that a module loading should be, as much as possible, "side
effects free", and thus shouldn't have temporary errors. But well, in
practice, module loading is typically the time where process-wide
initialization are done (modifying sys.path, os.environ, instantiating
connection or thread pools, registering atexit handler, starting
maintenance threads...), so that case has chances to happen at a moment or
another, especially if accesses to filesystem or network (SQL...) are done
at module loading, due to the lack of initialization system at upper
levels.
>
> That's why I propose modifying the behaviour of module import, so that
submodules are deleted as well when a parent module import fails. True, it
means they will be reloaded as well when importing the parent will start
again, but anyway we already have a "double execution" problem with the
reloading of the parent module, so it shouldn't make a big difference.
> The only other solution I'd see would be to SYSTEMATICALLY perform name
(re)binding when processing a from...import statement, to recover from the
previously failed initialization. Dunno if it's a good idea.
>
> On a (separate but related) topic, to be safer on module reimports or
reloadings, it could be interesting to add some "idempotency" to common
initialization tasks ; for example the "atexit" registration system,
wouldn't it be worth adding a boolean flag to explicitely skip registration
if a callable with same fully qualified name is already registered.
>
> Do you have opinions on these subjects ?

Back on topic...

As I stated on the issue, I think purging the whole subtree when a package
implicitly imports child modules is the least bad of the available options,
and better than leaving the child modules in place in violation of the "all
parent packages can be assumed to be in sys.modules" invariant (which is
what we do now).

Cheers,
Nick.
>
> thanks,
> regards,
> Pascal
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130703/c88841fd/attachment.html>