[issue33666] os.errno gone AWOL
New submission from Miro Hrončok <miro@hroncok.cz>: In 3.7.0b4 I see the following traceback:
import os os.errno Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: module 'os' has no attribute 'errno'
This was not the case for Python 3.6:
import os os.errno <module 'errno' (built-in)>
os.errno might not have been documented, however there are projects out there that use it: https://github.com/intel/bmap-tools/issues/34 https://bugzilla.redhat.com/show_bug.cgi?id=1583196 So I suggest the removal of os.errno is something worth documenting at https://docs.python.org/3.7/whatsnew/3.7.html The root of the change is in https://github.com/python/cpython/pull/1269/files#diff-405b29928f2a3ae216e45... ---------- assignee: docs@python components: Documentation, Library (Lib) messages: 317847 nosy: docs@python, hroncok priority: normal severity: normal status: open title: os.errno gone AWOL type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment: os.errno as well as os.sys or os.abc is an implementation detail. It can be changed without notice even in a bugfix release. Projects that depend on it are incorrect and should be fixed. ---------- nosy: +serhiy.storchaka resolution: -> not a bug stage: -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Petr Viktorin <encukou@gmail.com> added the comment: I think this was closed prematurely.
Projects that depend on it are incorrect and should be fixed.
That's definitely correct; no one here is arguing against it. However, there are projects out there depending on it -- it worked since around Python 2.5, and is readily available in `os.` tab completion. I think that documenting the removal would be good for our users. Furthermore, not all of its uses can be fixed. I've seen it printed in books. There are many answers/guides/tutorials around mentioning os.errno, for example: http://nullege.com/codes/search/os.errno.EACCES https://ubuntuforums.org/showthread.php?t=1459923 http://code.activestate.com/recipes/580759-show-os-error-codes-and-messages-... https://www.georgevreilly.com/blog/2016/03/24/RaisingIOErrorForFileNotFound.... ... and people tend based their code on such recipes, rather than checking what's documented. And, of course, projects run into this: https://github.com/intel/bmap-tools/issues/34 https://bugzilla.redhat.com/show_bug.cgi?id=1583196 https://github.com/python/typeshed/issues/1646 ---------- nosy: +petr.viktorin resolution: not a bug -> status: closed -> open _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
STINNER Victor <vstinner@redhat.com> added the comment:
os.errno as well as os.sys or os.abc is an implementation detail.
I agree but...
It can be changed without notice even in a bugfix release. Projects that depend on it are incorrect and should be fixed.
I don't think that it would hurt to document the removal of "os.errno" in "Porting to Python 3.7": https://docs.python.org/dev/whatsnew/3.7.html#changes-in-python-behavior Miro: are you volunteer to write such change? ---------- nosy: +vstinner _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Miro Hrončok <miro@hroncok.cz> added the comment: I can document this, yes. I've opened this issue so I could prep a PR. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment: If document disappearing of os.errno, we should document disappearing of all other names in other modules in all versions. It is possible to write a script that outputs dir(mod) for all modules and compare results between different Python versions. Are you volunteer to do this work? ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Change by STINNER Victor <vstinner@redhat.com>: ---------- title: os.errno gone AWOL -> os.errno gone absent without official leave (AWOL) _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
STINNER Victor <vstinner@redhat.com> added the comment:
If document disappearing of os.errno, we should document disappearing of all other names in other modules in all versions.
Miro proposed to document the removal of os.errno since errno is a commonly used module, as the os module. I don't think that the removal of other aliases are as important as that one. By the way, os.errno has been removed by bpo-30152: PR 1269. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Change by Petr Viktorin <encukou@gmail.com>: ---------- title: os.errno gone absent without official leave (AWOL) -> Document removal of os.errno _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment: Here is a list of all names removed in 3.7 (except modules that was removed as a whole): abc.WeakSet argparse._collections argparse._copy argparse._ensure_value argparse._textwrap ast._NUM_TYPES asyncio.async asyncio.base_events.compat asyncio.base_events.coroutine asyncio.base_events._ensure_resolved asyncio.base_events.inspect asyncio.base_events._is_dgram_socket asyncio.base_events._is_stream_socket asyncio.base_futures.events asyncio.base_subprocess.compat asyncio.base_subprocess.coroutine asyncio.coroutines._AwaitableABC asyncio.coroutines.compat asyncio.coroutines._CoroutineABC asyncio.coroutines.debug_wrapper asyncio.coroutines.events asyncio.coroutines._inspect_iscoroutinefunction asyncio.coroutines.opcode asyncio.coroutines._types_coroutine asyncio.coroutines._types_CoroutineType asyncio.coroutines._YIELD_FROM asyncio.coroutines._YIELD_FROM_BUG asyncio.events.compat asyncio.events.constants asyncio.events.extract_stack asyncio.events._format_args_and_kwargs asyncio.events._format_callback asyncio.events._format_callback_source asyncio.events.functools asyncio.events._get_function_source asyncio.events.inspect asyncio.events.reprlib asyncio.events.traceback asyncio.futures.compat asyncio.futures.traceback asyncio.futures._TracebackLogger asyncio.locks.compat asyncio.proactor_events.compat asyncio.queues.compat asyncio.queues.coroutine asyncio.selector_events.compat asyncio.selector_events.coroutine asyncio.selector_events._SelectorSslTransport asyncio.selectors asyncio.sslproto.compat asyncio.sslproto._is_sslproto_available asyncio.streams.compat asyncio.streams.coroutine asyncio.subprocess.coroutine asyncio.tasks.async asyncio.tasks.compat asyncio.transports.compat asyncio.unix_events.compat asyncio.unix_events.coroutine asyncio.unix_events._fspath asyncio.unix_events._set_inheritable asyncio.unix_events._set_nonblocking code.argparse collections.AsyncGenerator collections.AsyncIterable collections.AsyncIterator collections.Awaitable collections.ByteString collections.Callable collections._class_template collections.Collection collections.Container collections.Coroutine collections._field_template collections.Generator collections.Hashable collections.ItemsView collections.Iterable collections.Iterator collections.KeysView collections.Mapping collections.MappingView collections.MutableMapping collections.MutableSequence collections.MutableSet collections._repr_template collections.Reversible collections.Sequence collections.Set collections.Sized collections.ValuesView concurrent.futures.process.multiprocessing concurrent.futures.process._shutdown concurrent.futures.process.SimpleQueue concurrent.futures.process._threads_queues concurrent.futures.thread concurrent.futures.ThreadPoolExecutor distutils.cmd.install_misc distutils.command.sdist.AsyncGeneratorType distutils.command.sdist.BuiltinFunctionType distutils.command.sdist.BuiltinMethodType distutils.command.sdist.CCompilerError distutils.command.sdist.CodeType distutils.command.sdist.CompileError distutils.command.sdist.coroutine distutils.command.sdist.CoroutineType distutils.command.sdist.dep_util distutils.command.sdist.DistutilsArgError distutils.command.sdist.DistutilsByteCompileError distutils.command.sdist.DistutilsClassError distutils.command.sdist.DistutilsError distutils.command.sdist.DistutilsExecError distutils.command.sdist.DistutilsFileError distutils.command.sdist.DistutilsGetoptError distutils.command.sdist.DistutilsInternalError distutils.command.sdist.DistutilsModuleError distutils.command.sdist.DistutilsPlatformError distutils.command.sdist.DistutilsSetupError distutils.command.sdist.DynamicClassAttribute distutils.command.sdist.FrameType distutils.command.sdist.FunctionType distutils.command.sdist.GeneratorType distutils.command.sdist.GetSetDescriptorType distutils.command.sdist.LambdaType distutils.command.sdist.LibError distutils.command.sdist.LinkError distutils.command.sdist.MappingProxyType distutils.command.sdist.MemberDescriptorType distutils.command.sdist.MethodType distutils.command.sdist.ModuleType distutils.command.sdist.new_class distutils.command.sdist.prepare_class distutils.command.sdist.PreprocessError distutils.command.sdist.SimpleNamespace distutils.command.sdist.TracebackType distutils.command.sdist.UnknownFileError doctest.argparse email.utils.ecre enum._or_ enum.reduce functools.MappingProxyType functools.WeakKeyDictionary gettext.copy gettext.ENOENT gettext.io gettext.struct http.client.os http.cookies._warn_deprecated_setter http.server.argparse importlib._bootstrap_external._code_to_bytecode importlib._bootstrap_external._validate_bytecode_header inspect.ast json.tool.collections lib2to3.pgen2.grammar.tokenize lib2to3.pgen2.tokenize.t lib2to3.pytree.warnings lib2to3.refactor._from_system_newlines lib2to3.refactor._open_with_encoding lib2to3.refactor._to_system_newlines locale.collections logging.config.thread modulefinder.struct multiprocessing.forkserver.read_unsigned multiprocessing.forkserver.UNSIGNED_STRUCT multiprocessing.forkserver.write_unsigned multiprocessing.reduction.abstractmethod ntpath.splitunc os.errno os.stat_float_times pathlib.contextmanager plistlib.Dict plistlib._InternalDict plistlib.Plist profile.OptionParser profile.os re._alphanum_bytes re._alphanum_str re._pattern_type socketserver.errno sre_compile.RANGE_IGNORE sre_constants.RANGE_IGNORE sre_parse._parse_sub_cond sre_parse.RANGE_IGNORE ssl.ipaddress ssl.re ssl.textwrap statistics.chain statistics.decimal subprocess._PLATFORM_DEFAULT_CLOSE_FDS tabnanny.getopt tarfile.__cvsid__ tarfile.__date__ token.ASYNC token.AWAIT tokenize.ASYNC tokenize.AWAIT trace.argparse types._collections_abc types._functools typing._Any typing.CallableMeta typing._ClassVar typing._collections_abc typing.collections_abc typing._FinalTypingBase typing._ForwardRef typing._G_base typing.GenericMeta typing._generic_new typing._get_type_vars typing._make_subclasshook typing._next_in_mro typing._NoReturn typing._no_slots_copy typing._Optional typing._PY36 typing._qualname typing._replace_arg typing._subs_tree typing._trim_name typing.TupleMeta typing._TypeAlias typing._type_vars typing._TypingBase typing.TypingMeta typing._Union unittest.util._ordered_count unittest.util.OrderedDict urllib.request.collections uuid.ctypes uuid.lib uuid.libname uuid._unixdll_getnode uuid._uuid_generate_time weakref.collections zipfile.re ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment: And here is a list of names removed in 3.6: asynchat.fifo asyncio.coroutines.futures asyncio.futures.Error asyncio.futures.reprlib asyncio.tasks.linecache asyncio.tasks.traceback copy.builtins copy._copy_with_constructor copy._copy_with_copy_method copy._EmptyClass copy.name copy.PyStringMap copy.t copy.weakref datetime._divide_and_round distutils.command.config.sys distutils.command.register.os distutils.command.register.string distutils.command.sdist.string distutils.extension.sys distutils.text_file.os email.feedparser.message email.header._embeded_header email._header_value_parser._Folded email.message.warnings ensurepip._MISSING_SSL_MESSAGE ensurepip._require_ssl_for_pip ensurepip.ssl fileinput.DEFAULT_BUFSIZE ftplib.os ftplib.warnings glob.glob2 http.server._quote_html importlib._bootstrap_external._BACKCOMPAT_MAGIC_NUMBER importlib._bootstrap_external._verbose_message importlib._bootstrap.FileFinder importlib._bootstrap._ManageReload importlib._bootstrap._POPULATE importlib._bootstrap.SourceFileLoader importlib.util._BACKCOMPAT_MAGIC_NUMBER importlib.util._Module inspect.getmoduleinfo inspect.ModuleInfo lib2to3.fixer_util.islice lib2to3.fixes.fix_dict.ArgList lib2to3.fixes.fix_dict.LParen lib2to3.fixes.fix_dict.RParen lib2to3.fixes.fix_dict.token lib2to3.fixes.fix_exec.pytree lib2to3.fixes.fix_filter.Call lib2to3.fixes.fix_filter.token lib2to3.fixes.fix_has_key.token lib2to3.fixes.fix_metaclass.Name lib2to3.fixes.fix_nonzero.syms lib2to3.fixes.fix_print.is_tuple lib2to3.fixes.fix_types.token lib2to3.fixes.fix_urllib.fixer_base lib2to3.fixes.fix_zip.Call lib2to3.patcomp.os lib2to3.patcomp._PATTERN_GRAMMAR_FILE lib2to3.refactor.bu lib2to3.refactor.with_statement multiprocessing.connection.ForkingPickler multiprocessing.heap.context multiprocessing.managers.context multiprocessing.popen_forkserver.context multiprocessing.popen_spawn_posix.context multiprocessing.queues.ForkingPickler multiprocessing.sharedctypes.ForkingPickler multiprocessing.spawn.pickle os._DummyDirEntry os._dummy_scandir pathlib._cached pyclbr.itemgetter pyclbr.os _pyio.array re._cache_repl re.sys site.aliasmbcs site.CONFIG_LINE sre_parse.isdigit sre_parse.isident sre_parse.isname ssl._import_symbols statistics._decimal_to_ratio tarfile.TarIter trace._err_exit trace.find_executable_linenos trace.find_lines trace.find_lines_from_code trace.find_strings trace.fullmodname trace.Ignore trace.modname trace.rx_blank trace.usage trace._usage trace._warn turtledemo.bytedesign.math turtledemo.planet_and_moon.sleep typing._geqv typing._gorg unittest.mock.DescriptorTypes unittest.mock._slotted wsgiref.simple_server.BufferedWriter xml.etree.ElementTree._IterParseIterator ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Petr Viktorin <encukou@gmail.com> added the comment: Nothing there strikes me to be as prevalent or as discoverable as os.errno. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
INADA Naoki <songofacandy@gmail.com> added the comment: Even if it is documented, arn't people know it by running their code on Python 3.7? How the document help them? It's very easy to know `errno` module when find ImportError. And it's much easier than checking "waht's new" document. So I doubt it's worth. And if people see the document, people may think "all removed subimports should be documented" although only os.errno is special. If document os.errno removal, please note about all undocumented subimports are implementation detail and will be removed without any timing, even on micro version. (We will remove subimports for various reasons; avoiding huge unnecessary dependency, fixing regression caused by circular imports, etc...) ---------- nosy: +inada.naoki _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment: Actually removing re._pattern_type, uuid._uuid_generate_time and some private names in the typing module broke several third-party projects. See for example: https://github.com/search?q=_pattern_type&type=Issues https://github.com/search?q=_uuid_generate_time&type=Issues https://github.com/bintoro/overloading.py/issues/5 https://github.com/hsolbrig/PyShEx/issues/17 I'm sure there are examples for other removed names. I don't understand why os.errno should be special. Books can contain errors (e.g. the claim that Python 3 is not Turing complete). ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Petr Viktorin <encukou@gmail.com> added the comment:
Even if it is documented, arn't people know it by running their code on Python 3.7? How the document help them?
"What's new" is the right place to check if something breaks for you on Python 3.7. Let's make it useful.
It's very easy to know `errno` module when find ImportError. And it's much easier than checking "waht's new" document. So I doubt it's worth.
I disagree. Sure, some people will find answers on Stack Overflow or blog posts, but all those should link to official docs.
And if people see the document, people may think "all removed subimports should be documented" although only os.errno is special. If document os.errno removal, please note about all undocumented subimports are implementation detail and will be removed without any timing, even on micro version. (We will remove subimports for various reasons; avoiding huge unnecessary dependency, fixing regression caused by circular imports, etc...)
+1. Let's make that the main point. Something like the text below? Several undocumented internal imports were removed. One example is that `os.errno` is no longer available; use `import errno` directly instead. Note that such undocumented internal imports may be removed any time without notice, even in micro version releases. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Petr Viktorin <encukou@gmail.com> added the comment:
Actually removing re._pattern_type, uuid._uuid_generate_time and some private names in the typing module broke several third-party projects.
Those have underscores; that explicitly marks them as private/use at your own risk/will break in future versions. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
STINNER Victor <vstinner@redhat.com> added the comment: I like the idea of a generic sentence explaining that regularly we removed aliases to speedup Python startup time can remove aliases on purpose. msg317876: "Here is a list of all names removed in 3.7 (except modules that was removed as a whole): (... long list ...)" I agree that this list is very long. But I don't think that it would make sense to document all these removed symbols (aliases). IHMO it's more unlikely that someone gets the re module from "zipfile.re", than getting WeakSet from "abc.WeakSet". For "code.argparse", it's similar: the code module is a rarely used module, whereas argparse is a commonly used and known module. My comments on the list. Is already documented: * os.stat_float_times: https://docs.python.org/dev/whatsnew/3.7.html#api-and-feature-removals * collections.<many ABC classes>: https://docs.python.org/dev/whatsnew/3.7.html#id3 * token.ASYNC, token.AWAIT, tokenize.ASYNC, tokenize.AWAIT: they became keywords Should be documented: * os.errno: errno and os are tighly coupled, for example there is a os.strerror() function and OSError exception (named starting with "os") have an errno attribute May be documented: * inspect.ast: ast and inspect are tighly coupled to inspect code Should not be documented: * asyncio.* * distutils.* * concurrent.futures.* * gettext.* * code.argparse * doctest.argparse * profile.os * ssl.ipaddress * ssl.re * ssl.textwrap * statistics.decimal * tabnanny.getopt * zipfile.re * email.utils.ecre * functools.MappingProxyType * functools.WeakKeyDictionary * locale.collections * ... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
STINNER Victor <vstinner@redhat.com> added the comment: Petr Viktorin started a thread on python-ideas as well: https://mail.python.org/pipermail/python-ideas/2018-May/051098.html ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Change by INADA Naoki <songofacandy@gmail.com>: ---------- keywords: +patch pull_requests: +8015 stage: resolved -> patch review _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Petr Viktorin <encukou@gmail.com> added the comment: New changeset 1d2dafa249c7fb34f3d24e7a77d1bea02907d92b by Petr Viktorin (INADA Naoki) in branch 'master': bpo-33666: Add what's new entry for os.errno removal (GH-#8497) https://github.com/python/cpython/commit/1d2dafa249c7fb34f3d24e7a77d1bea0290... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- pull_requests: +8044 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Petr Viktorin <encukou@gmail.com> added the comment: New changeset 78c54de575fd121d91749c2316fef1949d76fb07 by Petr Viktorin (Miss Islington (bot)) in branch '3.7': bpo-33666: Add what's new entry for os.errno removal (GH-8497) (GH-8526) https://github.com/python/cpython/commit/78c54de575fd121d91749c2316fef1949d7... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
Change by INADA Naoki <songofacandy@gmail.com>: ---------- resolution: -> fixed stage: patch review -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue33666> _______________________________________
participants (6)
-
INADA Naoki -
Miro Hrončok
-
miss-islington
-
Petr Viktorin
-
Serhiy Storchaka
-
STINNER Victor