[Python-checkins] bpo-37330: open() no longer accept 'U' in file mode (GH-28118)
ambv
webhook-mailer at python.org
Thu Sep 2 06:58:05 EDT 2021
https://github.com/python/cpython/commit/19ba2122ac7313ac29207360cfa864a275b9489e
commit: 19ba2122ac7313ac29207360cfa864a275b9489e
branch: main
author: Victor Stinner <vstinner at python.org>
committer: ambv <lukasz at langa.pl>
date: 2021-09-02T12:58:00+02:00
summary:
bpo-37330: open() no longer accept 'U' in file mode (GH-28118)
open(), io.open(), codecs.open() and fileinput.FileInput no longer
accept "U" ("universal newline") in the file mode. This flag was
deprecated since Python 3.3.
files:
A Misc/NEWS.d/next/Core and Builtins/2021-09-02-01-28-01.bpo-37330.QDjM_l.rst
M Doc/library/codecs.rst
M Doc/library/fileinput.rst
M Doc/library/functions.rst
M Doc/whatsnew/3.11.rst
M Lib/_pyio.py
M Lib/fileinput.py
M Lib/imp.py
M Lib/test/test_codecs.py
M Lib/test/test_fileinput.py
M Lib/test/test_io.py
M Modules/_io/_iomodule.c
M Modules/_io/clinic/_iomodule.c.h
diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
index 0dcd88f9fd5b7..9ea689acd8bbb 100644
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -204,6 +204,9 @@ wider range of codecs when working with binary files:
*buffering* has the same meaning as for the built-in :func:`open` function.
It defaults to -1 which means that the default buffer size will be used.
+ .. versionchanged:: 3.11
+ The ``'U'`` mode has been removed.
+
.. function:: EncodedFile(file, data_encoding, file_encoding=None, errors='strict')
diff --git a/Doc/library/fileinput.rst b/Doc/library/fileinput.rst
index 3880ed3d2bfc9..9f7802dd4566d 100644
--- a/Doc/library/fileinput.rst
+++ b/Doc/library/fileinput.rst
@@ -153,7 +153,7 @@ available for subclassing as well:
and :meth:`~io.TextIOBase.readline` cannot be mixed.
With *mode* you can specify which file mode will be passed to :func:`open`. It
- must be one of ``'r'``, ``'rU'``, ``'U'`` and ``'rb'``.
+ must be one of ``'r'`` and ``'rb'``.
The *openhook*, when given, must be a function that takes two arguments,
*filename* and *mode*, and returns an accordingly opened file-like object. You
@@ -171,9 +171,6 @@ available for subclassing as well:
.. versionchanged:: 3.2
Can be used as a context manager.
- .. deprecated:: 3.4
- The ``'rU'`` and ``'U'`` modes.
-
.. deprecated:: 3.8
Support for :meth:`__getitem__` method is deprecated.
@@ -183,6 +180,9 @@ available for subclassing as well:
.. versionchanged:: 3.10
The keyword-only parameter *encoding* and *errors* are added.
+ .. versionchanged:: 3.11
+ The ``'rU'`` and ``'U'`` modes have been removed.
+
**Optional in-place filtering:** if the keyword argument ``inplace=True`` is
passed to :func:`fileinput.input` or to the :class:`FileInput` constructor, the
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
index 9629acec2bc72..a8fc7023d1195 100644
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -1156,12 +1156,6 @@ are always available. They are listed here in alphabetical order.
first decoded using a platform-dependent encoding or using the specified
*encoding* if given.
- There is an additional mode character permitted, ``'U'``, which no longer
- has any effect, and is considered deprecated. It previously enabled
- :term:`universal newlines` in text mode, which became the default behavior
- in Python 3.0. Refer to the documentation of the
- :ref:`newline <open-newline-parameter>` parameter for further details.
-
.. note::
Python doesn't depend on the underlying operating system's notion of text
@@ -1304,8 +1298,7 @@ are always available. They are listed here in alphabetical order.
The ``mode`` and ``flags`` arguments may have been modified or inferred from
the original call.
- .. versionchanged::
- 3.3
+ .. versionchanged:: 3.3
* The *opener* parameter was added.
* The ``'x'`` mode was added.
@@ -1313,30 +1306,26 @@ are always available. They are listed here in alphabetical order.
* :exc:`FileExistsError` is now raised if the file opened in exclusive
creation mode (``'x'``) already exists.
- .. versionchanged::
- 3.4
+ .. versionchanged:: 3.4
* The file is now non-inheritable.
- .. deprecated-removed:: 3.4 3.10
-
- The ``'U'`` mode.
-
- .. versionchanged::
- 3.5
+ .. versionchanged:: 3.5
* If the system call is interrupted and the signal handler does not raise an
exception, the function now retries the system call instead of raising an
:exc:`InterruptedError` exception (see :pep:`475` for the rationale).
* The ``'namereplace'`` error handler was added.
- .. versionchanged::
- 3.6
+ .. versionchanged:: 3.6
* Support added to accept objects implementing :class:`os.PathLike`.
* On Windows, opening a console buffer may return a subclass of
:class:`io.RawIOBase` other than :class:`io.FileIO`.
+ .. versionchanged:: 3.11
+ The ``'U'`` mode has been removed.
+
.. function:: ord(c)
Given a string representing one Unicode character, return an integer
diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst
index 7d42c840c4a19..d6a95a2e3175c 100644
--- a/Doc/whatsnew/3.11.rst
+++ b/Doc/whatsnew/3.11.rst
@@ -321,6 +321,14 @@ Changes in the Python API
Python 3.8.
(Contributed by Illia Volochii in :issue:`43234`.)
+* :func:`open`, :func:`io.open`, :func:`codecs.open` and
+ :class:`fileinput.FileInput` no longer accept ``'U'`` ("universal newline")
+ in the file mode. This flag was deprecated since Python 3.3. In Python 3, the
+ "universal newline" is used by default when a file is open in text mode. The
+ :ref:`newline parameter <open-newline-parameter>` of :func:`open` controls
+ how universal newlines works.
+ (Contributed by Victor Stinner in :issue:`37330`.)
+
C API Changes
=============
diff --git a/Lib/_pyio.py b/Lib/_pyio.py
index 56e9a0cb33c5f..d7119742b9d22 100644
--- a/Lib/_pyio.py
+++ b/Lib/_pyio.py
@@ -101,7 +101,6 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
'b' binary mode
't' text mode (default)
'+' open a disk file for updating (reading and writing)
- 'U' universal newline mode (deprecated)
========= ===============================================================
The default mode is 'rt' (open for reading text). For binary random
@@ -117,10 +116,6 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
returned as strings, the bytes having been first decoded using a
platform-dependent encoding or using the specified encoding if given.
- 'U' mode is deprecated and will raise an exception in future versions
- of Python. It has no effect in Python 3. Use newline to control
- universal newlines mode.
-
buffering is an optional integer used to set the buffering policy.
Pass 0 to switch buffering off (only allowed in binary mode), 1 to select
line buffering (only usable in text mode), and an integer > 1 to indicate
@@ -206,7 +201,7 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
if errors is not None and not isinstance(errors, str):
raise TypeError("invalid errors: %r" % errors)
modes = set(mode)
- if modes - set("axrwb+tU") or len(mode) > len(modes):
+ if modes - set("axrwb+t") or len(mode) > len(modes):
raise ValueError("invalid mode: %r" % mode)
creating = "x" in modes
reading = "r" in modes
@@ -215,13 +210,6 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
updating = "+" in modes
text = "t" in modes
binary = "b" in modes
- if "U" in modes:
- if creating or writing or appending or updating:
- raise ValueError("mode U cannot be combined with 'x', 'w', 'a', or '+'")
- import warnings
- warnings.warn("'U' mode is deprecated",
- DeprecationWarning, 2)
- reading = True
if text and binary:
raise ValueError("can't have text and binary mode at once")
if creating + reading + writing + appending > 1:
diff --git a/Lib/fileinput.py b/Lib/fileinput.py
index 35347185da048..d0b3caae5d6e4 100644
--- a/Lib/fileinput.py
+++ b/Lib/fileinput.py
@@ -217,15 +217,10 @@ def __init__(self, files=None, inplace=False, backup="", *,
EncodingWarning, 2)
# restrict mode argument to reading modes
- if mode not in ('r', 'rU', 'U', 'rb'):
- raise ValueError("FileInput opening mode must be one of "
- "'r', 'rU', 'U' and 'rb'")
- if 'U' in mode:
- import warnings
- warnings.warn("'U' mode is deprecated",
- DeprecationWarning, 2)
+ if mode not in ('r', 'rb'):
+ raise ValueError("FileInput opening mode must be 'r' or 'rb'")
self._mode = mode
- self._write_mode = mode.replace('r', 'w') if 'U' not in mode else 'w'
+ self._write_mode = mode.replace('r', 'w')
if openhook:
if inplace:
raise ValueError("FileInput cannot use an opening hook in inplace mode")
diff --git a/Lib/imp.py b/Lib/imp.py
index e02aaef344c61..71c5c8fc6a510 100644
--- a/Lib/imp.py
+++ b/Lib/imp.py
@@ -226,7 +226,7 @@ def load_module(name, file, filename, details):
"""
suffix, mode, type_ = details
- if mode and (not mode.startswith(('r', 'U')) or '+' in mode):
+ if mode and (not mode.startswith('r') or '+' in mode):
raise ValueError('invalid file open mode {!r}'.format(mode))
elif file is None and type_ in {PY_SOURCE, PY_COMPILED}:
msg = 'file object required for import (type code {})'.format(type_)
diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py
index 328a47b2e3766..f1a149f19b7d2 100644
--- a/Lib/test/test_codecs.py
+++ b/Lib/test/test_codecs.py
@@ -714,11 +714,23 @@ def test_bug691291(self):
self.addCleanup(os_helper.unlink, os_helper.TESTFN)
with open(os_helper.TESTFN, 'wb') as fp:
fp.write(s)
- with warnings_helper.check_warnings(('', DeprecationWarning)):
- reader = codecs.open(os_helper.TESTFN, 'U', encoding=self.encoding)
- with reader:
+ with codecs.open(os_helper.TESTFN, 'r',
+ encoding=self.encoding) as reader:
self.assertEqual(reader.read(), s1)
+ def test_invalid_modes(self):
+ for mode in ('U', 'rU', 'r+U'):
+ with self.assertRaises(ValueError) as cm:
+ codecs.open(os_helper.TESTFN, mode, encoding=self.encoding)
+ self.assertIn('invalid mode', str(cm.exception))
+
+ for mode in ('rt', 'wt', 'at', 'r+t'):
+ with self.assertRaises(ValueError) as cm:
+ codecs.open(os_helper.TESTFN, mode, encoding=self.encoding)
+ self.assertIn("can't have text and binary mode at once",
+ str(cm.exception))
+
+
class UTF16LETest(ReadTest, unittest.TestCase):
encoding = "utf-16-le"
ill_formed_sequence = b"\x80\xdc"
diff --git a/Lib/test/test_fileinput.py b/Lib/test/test_fileinput.py
index cae0eb1b5f657..31684aefdb0eb 100644
--- a/Lib/test/test_fileinput.py
+++ b/Lib/test/test_fileinput.py
@@ -230,20 +230,11 @@ def test_fileno(self):
line = list(fi)
self.assertEqual(fi.fileno(), -1)
- def test_opening_mode(self):
- try:
- # invalid mode, should raise ValueError
- fi = FileInput(mode="w", encoding="utf-8")
- self.fail("FileInput should reject invalid mode argument")
- except ValueError:
- pass
- # try opening in universal newline mode
- t1 = self.writeTmp(b"A\nB\r\nC\rD", mode="wb")
- with warnings_helper.check_warnings(('', DeprecationWarning)):
- fi = FileInput(files=t1, mode="U", encoding="utf-8")
- with warnings_helper.check_warnings(('', DeprecationWarning)):
- lines = list(fi)
- self.assertEqual(lines, ["A\n", "B\n", "C\n", "D"])
+ def test_invalid_opening_mode(self):
+ for mode in ('w', 'rU', 'U'):
+ with self.subTest(mode=mode):
+ with self.assertRaises(ValueError):
+ FileInput(mode=mode)
def test_stdin_binary_mode(self):
with mock.patch('sys.stdin') as m_stdin:
@@ -1015,10 +1006,6 @@ def check(mode, expected_lines):
self.assertEqual(lines, expected_lines)
check('r', ['A\n', 'B\n', 'C\n', 'D\u20ac'])
- with self.assertWarns(DeprecationWarning):
- check('rU', ['A\n', 'B\n', 'C\n', 'D\u20ac'])
- with self.assertWarns(DeprecationWarning):
- check('U', ['A\n', 'B\n', 'C\n', 'D\u20ac'])
with self.assertRaises(ValueError):
check('rb', ['A\n', 'B\r\n', 'C\r', 'D\u20ac'])
diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py
index 273545a2a2cbb..d52f97bb3965e 100644
--- a/Lib/test/test_io.py
+++ b/Lib/test/test_io.py
@@ -3954,16 +3954,6 @@ def test_attributes(self):
self.assertEqual(f.mode, "wb")
f.close()
- with warnings_helper.check_warnings(('', DeprecationWarning)):
- f = self.open(os_helper.TESTFN, "U", encoding="utf-8")
- self.assertEqual(f.name, os_helper.TESTFN)
- self.assertEqual(f.buffer.name, os_helper.TESTFN)
- self.assertEqual(f.buffer.raw.name, os_helper.TESTFN)
- self.assertEqual(f.mode, "U")
- self.assertEqual(f.buffer.mode, "rb")
- self.assertEqual(f.buffer.raw.mode, "rb")
- f.close()
-
f = self.open(os_helper.TESTFN, "w+", encoding="utf-8")
self.assertEqual(f.mode, "w+")
self.assertEqual(f.buffer.mode, "rb+") # Does it really matter?
@@ -3977,6 +3967,13 @@ def test_attributes(self):
f.close()
g.close()
+ def test_removed_u_mode(self):
+ # bpo-37330: The "U" mode has been removed in Python 3.11
+ for mode in ("U", "rU", "r+U"):
+ with self.assertRaises(ValueError) as cm:
+ self.open(os_helper.TESTFN, mode)
+ self.assertIn('invalid mode', str(cm.exception))
+
def test_open_pipe_with_append(self):
# bpo-27805: Ignore ESPIPE from lseek() in open().
r, w = os.pipe()
diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-09-02-01-28-01.bpo-37330.QDjM_l.rst b/Misc/NEWS.d/next/Core and Builtins/2021-09-02-01-28-01.bpo-37330.QDjM_l.rst
new file mode 100644
index 0000000000000..3f09449de70d0
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2021-09-02-01-28-01.bpo-37330.QDjM_l.rst
@@ -0,0 +1,4 @@
+:func:`open`, :func:`io.open`, :func:`codecs.open` and
+:class:`fileinput.FileInput` no longer accept ``'U'`` ("universal newline")
+in the file mode. This flag was deprecated since Python 3.3. Patch by Victor
+Stinner.
diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c
index 170dea41e8abd..b4743fbd5e04f 100644
--- a/Modules/_io/_iomodule.c
+++ b/Modules/_io/_iomodule.c
@@ -138,7 +138,6 @@ Character Meaning
'b' binary mode
't' text mode (default)
'+' open a disk file for updating (reading and writing)
-'U' universal newline mode (deprecated)
========= ===============================================================
The default mode is 'rt' (open for reading text). For binary random
@@ -154,10 +153,6 @@ bytes objects without any decoding. In text mode (the default, or when
returned as strings, the bytes having been first decoded using a
platform-dependent encoding or using the specified encoding if given.
-'U' mode is deprecated and will raise an exception in future versions
-of Python. It has no effect in Python 3. Use newline to control
-universal newlines mode.
-
buffering is an optional integer used to set the buffering policy.
Pass 0 to switch buffering off (only allowed in binary mode), 1 to select
line buffering (only usable in text mode), and an integer > 1 to indicate
@@ -233,12 +228,12 @@ static PyObject *
_io_open_impl(PyObject *module, PyObject *file, const char *mode,
int buffering, const char *encoding, const char *errors,
const char *newline, int closefd, PyObject *opener)
-/*[clinic end generated code: output=aefafc4ce2b46dc0 input=7295902222e6b311]*/
+/*[clinic end generated code: output=aefafc4ce2b46dc0 input=1543f4511d2356a5]*/
{
unsigned i;
int creating = 0, reading = 0, writing = 0, appending = 0, updating = 0;
- int text = 0, binary = 0, universal = 0;
+ int text = 0, binary = 0;
char rawmode[6], *m;
int line_buffering, is_number;
@@ -296,10 +291,6 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode,
case 'b':
binary = 1;
break;
- case 'U':
- universal = 1;
- reading = 1;
- break;
default:
goto invalid_mode;
}
@@ -322,18 +313,6 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode,
*m = '\0';
/* Parameters validation */
- if (universal) {
- if (creating || writing || appending || updating) {
- PyErr_SetString(PyExc_ValueError,
- "mode U cannot be combined with 'x', 'w', 'a', or '+'");
- goto error;
- }
- if (PyErr_WarnEx(PyExc_DeprecationWarning,
- "'U' mode is deprecated", 1) < 0)
- goto error;
- reading = 1;
- }
-
if (text && binary) {
PyErr_SetString(PyExc_ValueError,
"can't have text and binary mode at once");
diff --git a/Modules/_io/clinic/_iomodule.c.h b/Modules/_io/clinic/_iomodule.c.h
index 91c55b1816cd8..d5fb176eb66be 100644
--- a/Modules/_io/clinic/_iomodule.c.h
+++ b/Modules/_io/clinic/_iomodule.c.h
@@ -36,7 +36,6 @@ PyDoc_STRVAR(_io_open__doc__,
"\'b\' binary mode\n"
"\'t\' text mode (default)\n"
"\'+\' open a disk file for updating (reading and writing)\n"
-"\'U\' universal newline mode (deprecated)\n"
"========= ===============================================================\n"
"\n"
"The default mode is \'rt\' (open for reading text). For binary random\n"
@@ -52,10 +51,6 @@ PyDoc_STRVAR(_io_open__doc__,
"returned as strings, the bytes having been first decoded using a\n"
"platform-dependent encoding or using the specified encoding if given.\n"
"\n"
-"\'U\' mode is deprecated and will raise an exception in future versions\n"
-"of Python. It has no effect in Python 3. Use newline to control\n"
-"universal newlines mode.\n"
-"\n"
"buffering is an optional integer used to set the buffering policy.\n"
"Pass 0 to switch buffering off (only allowed in binary mode), 1 to select\n"
"line buffering (only usable in text mode), and an integer > 1 to indicate\n"
@@ -359,4 +354,4 @@ _io_open_code(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObjec
exit:
return return_value;
}
-/*[clinic end generated code: output=06e055d1d80b835d input=a9049054013a1b77]*/
+/*[clinic end generated code: output=6ea315343f6a94ba input=a9049054013a1b77]*/
More information about the Python-checkins
mailing list