Better error messages for missing optional stdlib packages
Hi all
When trying to import lzma on one of my machines, I was suprised to get a normal import error like for any other module. According to the docs lzma has been part of stdlib since 3.3. Further digging revealed that the error is due to the fact that xz wasn't compiled in when building Python. Since I suspect that there are other optional stdlib modules, this made me think whether the message in those cases should look a little more polished. Perhaps installing a stub module that prints some informative text before raising the relevant exception or similar. Also, maybe add a little note in the docs, stating that despite being part of stdlib this module might not be available on all systems.
Thanks, Marcus
On Wed, Oct 03, 2018 at 10:29:45PM +0200, Marcus Harnisch wrote:
Hi all
When trying to import lzma on one of my machines, I was suprised to get a normal import error like for any other module. According to the docs lzma has been part of stdlib since 3.3. Further digging revealed that the error is due to the fact that xz wasn't compiled in when building Python. Since I suspect that there are other optional stdlib modules, this made me think whether the message in those cases should look a little more polished. Perhaps installing a stub module that prints some informative text before raising the relevant exception or similar.
This sounds to me like something that the various Python distributors could do, e.g. Activestate, the Linux distros, etc. Especially since they're the ones compiling Python, they can control whether or not XY is supplied or not.
Also, maybe add a little note in the docs, stating that despite being part of stdlib this module might not be available on all systems.
That should be uncontroversial. Raise an issue on the bug tracker for that, or a patch on Github.
I agree with this. These small changes can save hundreds of people an hour each. In the long run, it pays off.
On Wednesday, October 3, 2018 at 4:31:08 PM UTC-4, Marcus Harnisch wrote:
Hi all
When trying to import lzma on one of my machines, I was suprised to get a normal import error like for any other module. According to the docs lzma has been part of stdlib since 3.3. Further digging revealed that the error is due to the fact that xz wasn't compiled in when building Python. Since I suspect that there are other optional stdlib modules, this made me think whether the message in those cases should look a little more polished. Perhaps installing a stub module that prints some informative text before raising the relevant exception or similar. Also, maybe add a little note in the docs, stating that despite being part of stdlib this module might not be available on all systems.
Thanks, Marcus
Python-ideas mailing list Python...@python.org javascript: https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Also, maybe add a little note in the docs, stating that despite being part of stdlib this module might not be available on all systems.
That should be uncontroversial. Raise an issue on the bug tracker for that, or a patch on Github.
I believe a PR that is more complex than a spelling fix needs a bug track number. At least that's what the PR submit form lead me to believe when I submitted a minor fix last week.
/ Anders
Hi Steven
On 10/04/2018 12:32 AM, Steven D'Aprano wrote:
On Wed, Oct 03, 2018 at 10:29:45PM +0200, Marcus Harnisch wrote:
When trying to import lzma on one of my machines, I was suprised to get a normal import error like for any other module. According to the docs lzma has been part of stdlib since 3.3. Further digging revealed that the error is due to the fact that xz wasn't compiled in when building Python. Since I suspect that there are other optional stdlib modules, this made me think whether the message in those cases should look a little more polished. Perhaps installing a stub module that prints some informative text before raising the relevant exception or similar.
This sounds to me like something that the various Python distributors could do, e.g. Activestate, the Linux distros, etc. Especially since they're the ones compiling Python, they can control whether or not XY is supplied or not.
I'd argue that it is Python that creates the dependency and is therefore responsible to handle this gracefully. But, I guess having this documented first is the most important step before discussing further measures.
Also, maybe add a little note in the docs, stating that despite being part of stdlib this module might not be available on all systems.
That should be uncontroversial. Raise an issue on the bug tracker for that, or a patch on Github.
Done. https://bugs.python.org/issue34895
Thanks, Marcus
On Thu, 4 Oct 2018 08:32:55 +1000 Steven D'Aprano steve@pearwood.info wrote:
Also, maybe add a little note in the docs, stating that despite being part of stdlib this module might not be available on all systems.
That should be uncontroversial. Raise an issue on the bug tracker for that, or a patch on Github.
I don't know. Realistically, any decent distributor of Python should include those optional modules, and I don't remember encountering a Python that doesn't has them. Adding a note saying otherwise in the docs may pointlessly scare users away.
Regards
Antoine.
On 10/04/2018 06:51 PM, Antoine Pitrou wrote:
I don't know. Realistically, any decent distributor of Python should include those optional modules, and I don't remember encountering a Python that doesn't has them.
What is the point, then, of making these packages optional in the first place? Shouldn't by that logic a missing dependency rather trigger a build failure /unless/ someone enables a certain switch (e.g. ‘--without-xz’), e.g. for special-purpose distribution.
Adding a note saying otherwise in the docs may pointlessly scare users away.
Possibly. But not saying anything isn't the best alternative under these circumstances.
Thanks, Marcus
On 10/3/2018 4:29 PM, Marcus Harnisch wrote:
When trying to import lzma on one of my machines, I was suprised to get a normal import error like for any other module.
What was the traceback and message? Did you get an import error for one of the three imports in lzma.py. I don't know why you would expect anything else. Any import in any stdlib module can potential fail if the file is buggy, corrupted, or missing.
According to the docs lzma has been part of stdlib since 3.3. Further digging revealed that the error is due to the fact that xz wasn't compiled in when building Python.
Perhaps this is a buggy build. Where did you get it from? Have you complained to the distributor?
lzma is documented as wrapping liblzma and supporting .xz files. If 'xz' is a separate library, then perhaps lzma should tolerate it missing.
Since I suspect that there are other optional stdlib modules,
I agree that modules that are necessarily optional should be documented as such, and as I mentioned on https://bugs.python.org/issue34895, many are so documented.
In the absence of such documentation, I would considered it to be not optional except as some distributor decides to omit it. But then it is the responsibility of the distributor to document the omission.
this made me think whether the message in those cases should look a little more polished. Perhaps installing a stub module that prints some informative text before raising the relevant exception or similar.
Also, maybe add a little note in the docs, stating that despite being part of stdlib this module might not be available on all systems.
ImportError messages and the documented status versus actual status of a module are two different issues. Let's keep them separate.
On 10/08/2018 12:29 AM, Terry Reedy wrote:
On 10/3/2018 4:29 PM, Marcus Harnisch wrote:
When trying to import lzma on one of my machines, I was suprised to get a normal import error like for any other module.
What was the traceback and message? Did you get an import error for one of the three imports in lzma.py. I don't know why you would expect anything else. Any import in any stdlib module can potential fail if the file is buggy, corrupted, or missing.
$ /usr/bin/python3 Python 3.7.0 (default, Oct 4 2018, 03:21:59) [GCC 8.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.
import lzma
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.7/lzma.py", line 27, in <module> ModuleNotFoundError: No module named '_lzma'
According to the docs lzma has been part of stdlib since 3.3. Further digging revealed that the error is due to the fact that xz wasn't compiled in when building Python.
Perhaps this is a buggy build.
This, I reckon, depends on the perspective and the definition of “buggy”. If the build process finishes without error, can we assume that the build is not buggy? If we make claims along the lines of “nobody in their right mind would build Python without lzma” it would only be fair to break the build if liblzma can't be detected. Unless I missed anything it doesn't happen until after the build has finished successfully, that a message is printed which lists the modules which couldn't be detected by setup.py. Here is a list of modules, which I believe are affected:
$ grep -F missing.append setup.py missing.append('spwd') missing.append('readline') missing.append('_ssl') missing.append('_hashlib') missing.append('_sqlite3') missing.append('_dbm') missing.append('_gdbm') missing.append('nis') missing.append('_curses') missing.append('_curses_panel') missing.append('zlib') missing.append('zlib') missing.append('zlib') missing.append('_bz2') missing.append('_lzma') missing.append('_elementtree') missing.append('ossaudiodev') missing.append('_tkinter') missing.append('_uuid')
Have you complained to the distributor?
After finding the root cause of the missing import I did file a request for including lzma in future releases of the distribution.
All I am asking is that unsuspecting users not be left in the dark when it comes to debugging unexpected import errors. I believe a missing stdlib module qualifies for “unexpected”. This could happen in form of documentation or by means of an import error handler that prints some helpful message in case that a stdlib module couldn't be found.
Regards, Marcus
participants (6)
-
Anders Hovmöller -
Antoine Pitrou -
Marcus Harnisch -
Neil Girdhar -
Steven D'Aprano -
Terry Reedy