[New-bugs-announce] [issue35486] subprocess module breaks backwards compatibility with older import hooks

David Wilson report at bugs.python.org
Thu Dec 13 18:14:15 EST 2018

New submission from David Wilson <dw at botanicus.net>:

The subprocess package since 880d42a3b24 / September 2018 has begun using this idiom:

    import _foo
except ModuleNotFoundError:

However, ModuleNotFoundError may not be thrown by older import hook implementations, since that subclass was only introduced in Python 3.6 -- and so the test above will fail. PEP-302 continues to document ImportError as the appropriate exception that should be raised.

https://mitogen.readthedocs.io/en/stable/ is one such import hook that lazily loads packages over the network when they aren't available locally. Current Python subprocess master breaks with Mitogen because when it discovers the master cannot satisfy the import, it throws ImportError.

The new exception subtype was introduced in https://bugs.python.org/issue15767 , however very little in the way of rationale was included, and so it's unclear to me what need the new subtype is addressing, whether this is a problem with the subprocess module or the subtype as a whole, or indeed whether any of this should be considered a bug.

It seems clear that some kind of regression is in the process of occurring during a minor release, and it also seems clear the new subtype will potentially spawn a whole variety of similar new regressions.

I will be updating Mitogen to throw the new subtype if it is available, but I felt it was important to report the regression to see what others think.

components: Library (Lib)
messages: 331774
nosy: dw
priority: normal
severity: normal
status: open
title: subprocess module breaks backwards compatibility with older import hooks
versions: Python 3.8

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list