Mailman 3 May 2019 - Python-checkins

[2.7] bpo-12639: msilib.Directory.start_component() fails if *keyfile* is not None (GH-13688)
by Steve Dower 31 May '19

31 May '19

https://github.com/python/cpython/commit/bfc1f605609218b9734d3cf3eab3531a2f… commit: bfc1f605609218b9734d3cf3eab3531a2f4624e1 branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: Steve Dower <steve.dower(a)python.org> date: 2019-05-31T15:39:39-07:00 summary: [2.7] bpo-12639: msilib.Directory.start_component() fails if *keyfile* is not None (GH-13688) * bpo-12639: msilib.Directory.start_component() fails if *keyfile* is not None (GH-13688) msilib.Directory.start_component() was passing an extra argument to CAB.gen_id(). (cherry picked from commit c8d5bf6c3fa09b43f6a5ee779d493d251dbcc53c) Co-authored-by: Zackery Spytz <zspytz(a)gmail.com> files: A Misc/NEWS.d/next/Library/2019-05-30-16-16-47.bpo-12639.TQFOR4.rst M Lib/msilib/__init__.py M Lib/test/test_msilib.py diff --git a/Lib/msilib/__init__.py b/Lib/msilib/__init__.py index 0352b60c0b25..9520dfc021a0 100644 --- a/Lib/msilib/__init__.py +++ b/Lib/msilib/__init__.py @@ -276,7 +276,7 @@ def start_component(self, component = None, feature = None, flags = None, keyfil if Win64: flags |= 256 if keyfile: - keyid = self.cab.gen_id(self.absolute, keyfile) + keyid = self.cab.gen_id(keyfile) self.keyfiles[keyfile] = keyid else: keyid = None diff --git a/Lib/test/test_msilib.py b/Lib/test/test_msilib.py index a2f3943e2281..58f709d70dcd 100644 --- a/Lib/test/test_msilib.py +++ b/Lib/test/test_msilib.py @@ -43,6 +43,14 @@ def test_summaryinfo_getproperty_issue1104(self): sum_info = None unlink(db_path) + def test_directory_start_component_keyfile(self): + db, db_path = init_database() + feature = msilib.Feature(db, 0, 'Feature', 'A feature', 'Python') + cab = msilib.CAB('CAB') + dir = msilib.Directory(db, cab, None, TESTFN, 'TARGETDIR', + 'SourceDir', 0) + dir.start_component(None, feature, None, 'keyfile') + class Test_make_id(unittest.TestCase): #http://msdn.microsoft.com/en-us/library/aa369212(v=vs.85).aspx diff --git a/Misc/NEWS.d/next/Library/2019-05-30-16-16-47.bpo-12639.TQFOR4.rst b/Misc/NEWS.d/next/Library/2019-05-30-16-16-47.bpo-12639.TQFOR4.rst new file mode 100644 index 000000000000..aade9121b4bb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-05-30-16-16-47.bpo-12639.TQFOR4.rst @@ -0,0 +1,2 @@ +:meth:`msilib.Directory.start_component()` no longer fails if *keyfile* is +not ``None``.

1 0

bpo-37105: Add deprecated-remove information on stream doc (#13672)
by Andrew Svetlov 31 May '19

31 May '19

https://github.com/python/cpython/commit/ed9f3562b637a59b9000abbceee5ae369d… commit: ed9f3562b637a59b9000abbceee5ae369d35444d branch: master author: Emmanuel Arias <emmanuelarias30(a)gmail.com> committer: Andrew Svetlov <andrew.svetlov(a)gmail.com> date: 2019-05-31T23:48:57+03:00 summary: bpo-37105: Add deprecated-remove information on stream doc (#13672) * Add deprecated-remove information on stream doc According to the code on streams.py the functions: ``open_connection()``, ``start_server()``, ``open_unix_connection()``, ``start_unix_server()`` are deprecated. I infor that on documentation. files: M Doc/library/asyncio-stream.rst diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index e735b81f234d..28ca5d5f3396 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -67,6 +67,10 @@ and work with streams: The *ssl_handshake_timeout* parameter. + .. deprecated-removed:: 3.8 3.10 + + `open_connection()` is deprecated in favor of `connect()`. + .. coroutinefunction:: start_server(client_connected_cb, host=None, \ port=None, \*, loop=None, limit=None, \ family=socket.AF_UNSPEC, \ @@ -100,6 +104,10 @@ and work with streams: The *ssl_handshake_timeout* and *start_serving* parameters. + .. deprecated-removed:: 3.8 3.10 + + `start_server()` is deprecated if favor of `StreamServer()` + .. rubric:: Unix Sockets @@ -124,6 +132,10 @@ and work with streams: The *path* parameter can now be a :term:`path-like object` + .. deprecated-removed:: 3.8 3.10 + + `open_unix_connection()` is deprecated if favor of `connect_unix()`. + .. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \ \*, loop=None, limit=None, sock=None, \ @@ -146,6 +158,10 @@ and work with streams: The *path* parameter can now be a :term:`path-like object`. + .. deprecated-removed:: 3.8 3.10 + + `start_unix_server()` is deprecated in favor of `UnixStreamServer()`. + ---------

1 0

bpo-15115: Document deprecation of email.encoders in Python 3 (GH-5354)
by Miss Islington (bot) 31 May '19

31 May '19

https://github.com/python/cpython/commit/464c1ec65af2c1c1d849d50d9726fa4538… commit: 464c1ec65af2c1c1d849d50d9726fa453804e70e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-05-31T13:26:01-07:00 summary: bpo-15115: Document deprecation of email.encoders in Python 3 (GH-5354) (cherry picked from commit a747c3a5edf21fa5670bc30f5e1d804de89ebf62) Co-authored-by: Cheryl Sabella <cheryl.sabella(a)gmail.com> files: M Doc/library/email.encoders.rst diff --git a/Doc/library/email.encoders.rst b/Doc/library/email.encoders.rst index e24ac7b90d29..debd1c85d490 100644 --- a/Doc/library/email.encoders.rst +++ b/Doc/library/email.encoders.rst @@ -12,6 +12,11 @@ This module is part of the legacy (``Compat32``) email API. In the new API the functionality is provided by the *cte* parameter of the :meth:`~email.message.EmailMessage.set_content` method. +This module is deprecated in Python 3. The functions provided here +should not be called explicitly since the :class:`~email.mime.text.MIMEText` +class sets the content type and CTE header using the *_subtype* and *_charset* +values passed during the instaniation of that class. + The remaining text in this section is the original documentation of the module. When creating :class:`~email.message.Message` objects from scratch, you often

1 0

bpo-15115: Document deprecation of email.encoders in Python 3 (GH-5354)
by Cheryl Sabella 31 May '19

31 May '19

https://github.com/python/cpython/commit/a747c3a5edf21fa5670bc30f5e1d804de8… commit: a747c3a5edf21fa5670bc30f5e1d804de89ebf62 branch: master author: Cheryl Sabella <cheryl.sabella(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-05-31T16:18:41-04:00 summary: bpo-15115: Document deprecation of email.encoders in Python 3 (GH-5354) files: M Doc/library/email.encoders.rst diff --git a/Doc/library/email.encoders.rst b/Doc/library/email.encoders.rst index 70bf61323c39..e4752a5edf84 100644 --- a/Doc/library/email.encoders.rst +++ b/Doc/library/email.encoders.rst @@ -12,6 +12,11 @@ This module is part of the legacy (``Compat32``) email API. In the new API the functionality is provided by the *cte* parameter of the :meth:`~email.message.EmailMessage.set_content` method. +This module is deprecated in Python 3. The functions provided here +should not be called explicitly since the :class:`~email.mime.text.MIMEText` +class sets the content type and CTE header using the *_subtype* and *_charset* +values passed during the instaniation of that class. + The remaining text in this section is the original documentation of the module. When creating :class:`~email.message.Message` objects from scratch, you often

1 0

bpo-33361: Fix bug with seeking in StreamRecoders (GH-8278)
by Berker Peksag 31 May '19

31 May '19

https://github.com/python/cpython/commit/a6dc5d4e1c9ef465dc1f1ad95c382aa8e3… commit: a6dc5d4e1c9ef465dc1f1ad95c382aa8e32b178f branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: Berker Peksag <berker.peksag(a)gmail.com> date: 2019-05-31T23:03:22+03:00 summary: bpo-33361: Fix bug with seeking in StreamRecoders (GH-8278) (cherry picked from commit a6ec1ce1ac05b1258931422e96eac215b6a05459) Co-authored-by: Ammar Askar <ammar_askar(a)hotmail.com> files: A Misc/NEWS.d/next/Library/2018-07-13-20-17-17.bpo-33361.dx2NVn.rst M Lib/codecs.py M Lib/test/test_codecs.py diff --git a/Lib/codecs.py b/Lib/codecs.py index 3cd78fc9f197..cfca5d38b079 100644 --- a/Lib/codecs.py +++ b/Lib/codecs.py @@ -847,6 +847,12 @@ def reset(self): self.reader.reset() self.writer.reset() + def seek(self, offset, whence=0): + # Seeks must be propagated to both the readers and writers + # as they might need to reset their internal buffers. + self.reader.seek(offset, whence) + self.writer.seek(offset, whence) + def __getattr__(self, name, getattr=getattr): diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index a3a2f6563a1f..248ef6fe07dd 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -3318,6 +3318,31 @@ def test_write(self): sr.write(text.encode('latin1')) self.assertEqual(bio.getvalue(), text.encode('utf-8')) + def test_seeking_read(self): + bio = io.BytesIO('line1\nline2\nline3\n'.encode('utf-16-le')) + sr = codecs.EncodedFile(bio, 'utf-8', 'utf-16-le') + + self.assertEqual(sr.readline(), b'line1\n') + sr.seek(0) + self.assertEqual(sr.readline(), b'line1\n') + self.assertEqual(sr.readline(), b'line2\n') + self.assertEqual(sr.readline(), b'line3\n') + self.assertEqual(sr.readline(), b'') + + def test_seeking_write(self): + bio = io.BytesIO('123456789\n'.encode('utf-16-le')) + sr = codecs.EncodedFile(bio, 'utf-8', 'utf-16-le') + + # Test that seek() only resets its internal buffer when offset + # and whence are zero. + sr.seek(2) + sr.write(b'\nabc\n') + self.assertEqual(sr.readline(), b'789\n') + sr.seek(0) + self.assertEqual(sr.readline(), b'1\n') + self.assertEqual(sr.readline(), b'abc\n') + self.assertEqual(sr.readline(), b'789\n') + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/Library/2018-07-13-20-17-17.bpo-33361.dx2NVn.rst b/Misc/NEWS.d/next/Library/2018-07-13-20-17-17.bpo-33361.dx2NVn.rst new file mode 100644 index 000000000000..2b71095984a0 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-13-20-17-17.bpo-33361.dx2NVn.rst @@ -0,0 +1,2 @@ +Fix a bug in :class:`codecs.StreamRecoder` where seeking might leave old data in a +buffer and break subsequent read calls. Patch by Ammar Askar.

1 0

bpo-33361: Fix bug with seeking in StreamRecoders (GH-8278)
by Berker Peksag 31 May '19

31 May '19

https://github.com/python/cpython/commit/a6ec1ce1ac05b1258931422e96eac215b6… commit: a6ec1ce1ac05b1258931422e96eac215b6a05459 branch: master author: Ammar Askar <ammar_askar(a)hotmail.com> committer: Berker Peksag <berker.peksag(a)gmail.com> date: 2019-05-31T22:44:00+03:00 summary: bpo-33361: Fix bug with seeking in StreamRecoders (GH-8278) files: A Misc/NEWS.d/next/Library/2018-07-13-20-17-17.bpo-33361.dx2NVn.rst M Lib/codecs.py M Lib/test/test_codecs.py diff --git a/Lib/codecs.py b/Lib/codecs.py index 884be0b2c02e..21c45a7d10a4 100644 --- a/Lib/codecs.py +++ b/Lib/codecs.py @@ -847,6 +847,12 @@ def reset(self): self.reader.reset() self.writer.reset() + def seek(self, offset, whence=0): + # Seeks must be propagated to both the readers and writers + # as they might need to reset their internal buffers. + self.reader.seek(offset, whence) + self.writer.seek(offset, whence) + def __getattr__(self, name, getattr=getattr): diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index f665febfc90a..47df88cedaaf 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -3166,6 +3166,31 @@ def test_write(self): sr.write(text.encode('latin1')) self.assertEqual(bio.getvalue(), text.encode('utf-8')) + def test_seeking_read(self): + bio = io.BytesIO('line1\nline2\nline3\n'.encode('utf-16-le')) + sr = codecs.EncodedFile(bio, 'utf-8', 'utf-16-le') + + self.assertEqual(sr.readline(), b'line1\n') + sr.seek(0) + self.assertEqual(sr.readline(), b'line1\n') + self.assertEqual(sr.readline(), b'line2\n') + self.assertEqual(sr.readline(), b'line3\n') + self.assertEqual(sr.readline(), b'') + + def test_seeking_write(self): + bio = io.BytesIO('123456789\n'.encode('utf-16-le')) + sr = codecs.EncodedFile(bio, 'utf-8', 'utf-16-le') + + # Test that seek() only resets its internal buffer when offset + # and whence are zero. + sr.seek(2) + sr.write(b'\nabc\n') + self.assertEqual(sr.readline(), b'789\n') + sr.seek(0) + self.assertEqual(sr.readline(), b'1\n') + self.assertEqual(sr.readline(), b'abc\n') + self.assertEqual(sr.readline(), b'789\n') + @unittest.skipIf(_testcapi is None, 'need _testcapi module') class LocaleCodecTest(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2018-07-13-20-17-17.bpo-33361.dx2NVn.rst b/Misc/NEWS.d/next/Library/2018-07-13-20-17-17.bpo-33361.dx2NVn.rst new file mode 100644 index 000000000000..2b71095984a0 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-07-13-20-17-17.bpo-33361.dx2NVn.rst @@ -0,0 +1,2 @@ +Fix a bug in :class:`codecs.StreamRecoder` where seeking might leave old data in a +buffer and break subsequent read calls. Patch by Ammar Askar.

1 0

bpo-26826: Expose copy_file_range in the os module (GH-7255)
by Pablo Galindo 31 May '19

31 May '19

https://github.com/python/cpython/commit/aac4d0342c3e692731c189d003dbd73a8c… commit: aac4d0342c3e692731c189d003dbd73a8c681a34 branch: master author: Pablo Galindo <Pablogsal(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-05-31T19:39:47+01:00 summary: bpo-26826: Expose copy_file_range in the os module (GH-7255) files: A Misc/NEWS.d/next/Core and Builtins/2018-05-30-23-43-03.bpo-26826.NkRzjb.rst M Doc/library/os.rst M Lib/test/test_os.py M Modules/clinic/posixmodule.c.h M Modules/posixmodule.c M aclocal.m4 M configure M configure.ac M pyconfig.h.in diff --git a/Doc/library/os.rst b/Doc/library/os.rst index b53fd71e65b3..107764ba4d53 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -707,6 +707,28 @@ as internal buffering of data. pass +.. function:: copy_file_range(src, dst, count, offset_src=None, offset_dst=None) + + Copy *count* bytes from file descriptor *src*, starting from offset + *offset_src*, to file descriptor *dst*, starting from offset *offset_dst*. + If *offset_src* is None, then *src* is read from the current position; + respectively for *offset_dst*. The files pointed by *src* and *dst* + must reside in the same filesystem, otherwise an :exc:`OSError` is + raised with :attr:`~OSError.errno` set to :data:`errno.EXDEV`. + + This copy is done without the additional cost of transferring data + from the kernel to user space and then back into the kernel. Additionally, + some filesystems could implement extra optimizations. The copy is done as if + both files are opened as binary. + + The return value is the amount of bytes copied. This could be less than the + amount requested. + + .. availability:: Linux kernel >= 4.5 or glibc >= 2.27. + + .. versionadded:: 3.8 + + .. function:: device_encoding(fd) Return a string describing the encoding of the device associated with *fd* diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index f17a19a7585d..a8eae6162057 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -231,6 +231,89 @@ def test_symlink_keywords(self): except (NotImplementedError, OSError): pass # No OS support or unprivileged user + @unittest.skipUnless(hasattr(os, 'copy_file_range'), 'test needs os.copy_file_range()') + def test_copy_file_range_invalid_values(self): + with self.assertRaises(ValueError): + os.copy_file_range(0, 1, -10) + + @unittest.skipUnless(hasattr(os, 'copy_file_range'), 'test needs os.copy_file_range()') + def test_copy_file_range(self): + TESTFN2 = support.TESTFN + ".3" + data = b'0123456789' + + create_file(support.TESTFN, data) + self.addCleanup(support.unlink, support.TESTFN) + + in_file = open(support.TESTFN, 'rb') + self.addCleanup(in_file.close) + in_fd = in_file.fileno() + + out_file = open(TESTFN2, 'w+b') + self.addCleanup(support.unlink, TESTFN2) + self.addCleanup(out_file.close) + out_fd = out_file.fileno() + + try: + i = os.copy_file_range(in_fd, out_fd, 5) + except OSError as e: + # Handle the case in which Python was compiled + # in a system with the syscall but without support + # in the kernel. + if e.errno != errno.ENOSYS: + raise + self.skipTest(e) + else: + # The number of copied bytes can be less than + # the number of bytes originally requested. + self.assertIn(i, range(0, 6)); + + with open(TESTFN2, 'rb') as in_file: + self.assertEqual(in_file.read(), data[:i]) + + @unittest.skipUnless(hasattr(os, 'copy_file_range'), 'test needs os.copy_file_range()') + def test_copy_file_range_offset(self): + TESTFN4 = support.TESTFN + ".4" + data = b'0123456789' + bytes_to_copy = 6 + in_skip = 3 + out_seek = 5 + + create_file(support.TESTFN, data) + self.addCleanup(support.unlink, support.TESTFN) + + in_file = open(support.TESTFN, 'rb') + self.addCleanup(in_file.close) + in_fd = in_file.fileno() + + out_file = open(TESTFN4, 'w+b') + self.addCleanup(support.unlink, TESTFN4) + self.addCleanup(out_file.close) + out_fd = out_file.fileno() + + try: + i = os.copy_file_range(in_fd, out_fd, bytes_to_copy, + offset_src=in_skip, + offset_dst=out_seek) + except OSError as e: + # Handle the case in which Python was compiled + # in a system with the syscall but without support + # in the kernel. + if e.errno != errno.ENOSYS: + raise + self.skipTest(e) + else: + # The number of copied bytes can be less than + # the number of bytes originally requested. + self.assertIn(i, range(0, bytes_to_copy+1)); + + with open(TESTFN4, 'rb') as in_file: + read = in_file.read() + # seeked bytes (5) are zero'ed + self.assertEqual(read[:out_seek], b'\x00'*out_seek) + # 012 are skipped (in_skip) + # 345678 are copied in the file (in_skip + bytes_to_copy) + self.assertEqual(read[out_seek:], + data[in_skip:in_skip+i]) # Test attributes on return values from os.*stat* family. class StatAttributeTests(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-05-30-23-43-03.bpo-26826.NkRzjb.rst b/Misc/NEWS.d/next/Core and Builtins/2018-05-30-23-43-03.bpo-26826.NkRzjb.rst new file mode 100644 index 000000000000..27d7f82a672d --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2018-05-30-23-43-03.bpo-26826.NkRzjb.rst @@ -0,0 +1 @@ +Expose :func:`copy_file_range` as a low level API in the :mod:`os` module. diff --git a/Modules/clinic/posixmodule.c.h b/Modules/clinic/posixmodule.c.h index 13f25460b4f6..22cb94761de5 100644 --- a/Modules/clinic/posixmodule.c.h +++ b/Modules/clinic/posixmodule.c.h @@ -5395,6 +5395,108 @@ os_pwritev(PyObject *module, PyObject *const *args, Py_ssize_t nargs) #endif /* (defined(HAVE_PWRITEV) || defined (HAVE_PWRITEV2)) */ +#if defined(HAVE_COPY_FILE_RANGE) + +PyDoc_STRVAR(os_copy_file_range__doc__, +"copy_file_range($module, /, src, dst, count, offset_src=None,\n" +" offset_dst=None)\n" +"--\n" +"\n" +"Copy count bytes from one file descriptor to another.\n" +"\n" +" src\n" +" Source file descriptor.\n" +" dst\n" +" Destination file descriptor.\n" +" count\n" +" Number of bytes to copy.\n" +" offset_src\n" +" Starting offset in src.\n" +" offset_dst\n" +" Starting offset in dst.\n" +"\n" +"If offset_src is None, then src is read from the current position;\n" +"respectively for offset_dst."); + +#define OS_COPY_FILE_RANGE_METHODDEF \ + {"copy_file_range", (PyCFunction)(void(*)(void))os_copy_file_range, METH_FASTCALL|METH_KEYWORDS, os_copy_file_range__doc__}, + +static PyObject * +os_copy_file_range_impl(PyObject *module, int src, int dst, Py_ssize_t count, + PyObject *offset_src, PyObject *offset_dst); + +static PyObject * +os_copy_file_range(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"src", "dst", "count", "offset_src", "offset_dst", NULL}; + static _PyArg_Parser _parser = {NULL, _keywords, "copy_file_range", 0}; + PyObject *argsbuf[5]; + Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 3; + int src; + int dst; + Py_ssize_t count; + PyObject *offset_src = Py_None; + PyObject *offset_dst = Py_None; + + args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 3, 5, 0, argsbuf); + if (!args) { + goto exit; + } + if (PyFloat_Check(args[0])) { + PyErr_SetString(PyExc_TypeError, + "integer argument expected, got float" ); + goto exit; + } + src = _PyLong_AsInt(args[0]); + if (src == -1 && PyErr_Occurred()) { + goto exit; + } + if (PyFloat_Check(args[1])) { + PyErr_SetString(PyExc_TypeError, + "integer argument expected, got float" ); + goto exit; + } + dst = _PyLong_AsInt(args[1]); + if (dst == -1 && PyErr_Occurred()) { + goto exit; + } + if (PyFloat_Check(args[2])) { + PyErr_SetString(PyExc_TypeError, + "integer argument expected, got float" ); + goto exit; + } + { + Py_ssize_t ival = -1; + PyObject *iobj = PyNumber_Index(args[2]); + if (iobj != NULL) { + ival = PyLong_AsSsize_t(iobj); + Py_DECREF(iobj); + } + if (ival == -1 && PyErr_Occurred()) { + goto exit; + } + count = ival; + } + if (!noptargs) { + goto skip_optional_pos; + } + if (args[3]) { + offset_src = args[3]; + if (!--noptargs) { + goto skip_optional_pos; + } + } + offset_dst = args[4]; +skip_optional_pos: + return_value = os_copy_file_range_impl(module, src, dst, count, offset_src, offset_dst); + +exit: + return return_value; +} + +#endif /* defined(HAVE_COPY_FILE_RANGE) */ + #if defined(HAVE_MKFIFO) PyDoc_STRVAR(os_mkfifo__doc__, @@ -8460,6 +8562,10 @@ os__remove_dll_directory(PyObject *module, PyObject *const *args, Py_ssize_t nar #define OS_PWRITEV_METHODDEF #endif /* !defined(OS_PWRITEV_METHODDEF) */ +#ifndef OS_COPY_FILE_RANGE_METHODDEF + #define OS_COPY_FILE_RANGE_METHODDEF +#endif /* !defined(OS_COPY_FILE_RANGE_METHODDEF) */ + #ifndef OS_MKFIFO_METHODDEF #define OS_MKFIFO_METHODDEF #endif /* !defined(OS_MKFIFO_METHODDEF) */ @@ -8635,4 +8741,4 @@ os__remove_dll_directory(PyObject *module, PyObject *const *args, Py_ssize_t nar #ifndef OS__REMOVE_DLL_DIRECTORY_METHODDEF #define OS__REMOVE_DLL_DIRECTORY_METHODDEF #endif /* !defined(OS__REMOVE_DLL_DIRECTORY_METHODDEF) */ -/*[clinic end generated code: output=855b81aafd05beed input=a9049054013a1b77]*/ +/*[clinic end generated code: output=b3ae8afd275ea5cd input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 77a3700ab22e..8f6cffffcdfb 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -117,6 +117,10 @@ corresponding Unix manual entries for more information on calls."); #include <sched.h> #endif +#ifdef HAVE_COPY_FILE_RANGE +#include <unistd.h> +#endif + #if !defined(CPU_ALLOC) && defined(HAVE_SCHED_SETAFFINITY) #undef HAVE_SCHED_SETAFFINITY #endif @@ -9455,8 +9459,74 @@ os_pwritev_impl(PyObject *module, int fd, PyObject *buffers, Py_off_t offset, } #endif /* HAVE_PWRITEV */ +#ifdef HAVE_COPY_FILE_RANGE +/*[clinic input] + +os.copy_file_range + src: int + Source file descriptor. + dst: int + Destination file descriptor. + count: Py_ssize_t + Number of bytes to copy. + offset_src: object = None + Starting offset in src. + offset_dst: object = None + Starting offset in dst. + +Copy count bytes from one file descriptor to another. + +If offset_src is None, then src is read from the current position; +respectively for offset_dst. +[clinic start generated code]*/ + +static PyObject * +os_copy_file_range_impl(PyObject *module, int src, int dst, Py_ssize_t count, + PyObject *offset_src, PyObject *offset_dst) +/*[clinic end generated code: output=1a91713a1d99fc7a input=42fdce72681b25a9]*/ +{ + off_t offset_src_val, offset_dst_val; + off_t *p_offset_src = NULL; + off_t *p_offset_dst = NULL; + Py_ssize_t ret; + int async_err = 0; + /* The flags argument is provided to allow + * for future extensions and currently must be to 0. */ + int flags = 0; + + + if (count < 0) { + PyErr_SetString(PyExc_ValueError, "negative value for 'count' not allowed"); + return NULL; + } + + if (offset_src != Py_None) { + if (!Py_off_t_converter(offset_src, &offset_src_val)) { + return NULL; + } + p_offset_src = &offset_src_val; + } + if (offset_dst != Py_None) { + if (!Py_off_t_converter(offset_dst, &offset_dst_val)) { + return NULL; + } + p_offset_dst = &offset_dst_val; + } + do { + Py_BEGIN_ALLOW_THREADS + ret = copy_file_range(src, p_offset_src, dst, p_offset_dst, count, flags); + Py_END_ALLOW_THREADS + } while (ret < 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); + + if (ret < 0) { + return (!async_err) ? posix_error() : NULL; + } + + return PyLong_FromSsize_t(ret); +} +#endif /* HAVE_COPY_FILE_RANGE*/ #ifdef HAVE_MKFIFO /*[clinic input] @@ -13432,6 +13502,7 @@ static PyMethodDef posix_methods[] = { OS_POSIX_SPAWN_METHODDEF OS_POSIX_SPAWNP_METHODDEF OS_READLINK_METHODDEF + OS_COPY_FILE_RANGE_METHODDEF OS_RENAME_METHODDEF OS_REPLACE_METHODDEF OS_RMDIR_METHODDEF diff --git a/aclocal.m4 b/aclocal.m4 index 85f00dd5fac7..3d6b1a375fdc 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -12,9 +12,9 @@ # PARTICULAR PURPOSE. m4_ifndef([AC_CONFIG_MACRO_DIRS], [m4_defun([_AM_CONFIG_MACRO_DIRS], [])m4_defun([AC_CONFIG_MACRO_DIRS], [_AM_CONFIG_MACRO_DIRS($@)])]) -dnl pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*- -dnl serial 11 (pkg-config-0.29.1) -dnl +# pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*- +# serial 11 (pkg-config-0.29.1) + dnl Copyright © 2004 Scott James Remnant <scott(a)netsplit.com>. dnl Copyright © 2012-2015 Dan Nicholson <dbn.lists(a)gmail.com> dnl @@ -288,5 +288,73 @@ AS_VAR_COPY([$1], [pkg_cv_][$1]) AS_VAR_IF([$1], [""], [$5], [$4])dnl ])dnl PKG_CHECK_VAR +dnl PKG_WITH_MODULES(VARIABLE-PREFIX, MODULES, +dnl [ACTION-IF-FOUND],[ACTION-IF-NOT-FOUND], +dnl [DESCRIPTION], [DEFAULT]) +dnl ------------------------------------------ +dnl +dnl Prepare a "--with-" configure option using the lowercase +dnl [VARIABLE-PREFIX] name, merging the behaviour of AC_ARG_WITH and +dnl PKG_CHECK_MODULES in a single macro. +AC_DEFUN([PKG_WITH_MODULES], +[ +m4_pushdef([with_arg], m4_tolower([$1])) + +m4_pushdef([description], + [m4_default([$5], [build with ]with_arg[ support])]) + +m4_pushdef([def_arg], [m4_default([$6], [auto])]) +m4_pushdef([def_action_if_found], [AS_TR_SH([with_]with_arg)=yes]) +m4_pushdef([def_action_if_not_found], [AS_TR_SH([with_]with_arg)=no]) + +m4_case(def_arg, + [yes],[m4_pushdef([with_without], [--without-]with_arg)], + [m4_pushdef([with_without],[--with-]with_arg)]) + +AC_ARG_WITH(with_arg, + AS_HELP_STRING(with_without, description[ @<:@default=]def_arg[@:>@]),, + [AS_TR_SH([with_]with_arg)=def_arg]) + +AS_CASE([$AS_TR_SH([with_]with_arg)], + [yes],[PKG_CHECK_MODULES([$1],[$2],$3,$4)], + [auto],[PKG_CHECK_MODULES([$1],[$2], + [m4_n([def_action_if_found]) $3], + [m4_n([def_action_if_not_found]) $4])]) + +m4_popdef([with_arg]) +m4_popdef([description]) +m4_popdef([def_arg]) + +])dnl PKG_WITH_MODULES + +dnl PKG_HAVE_WITH_MODULES(VARIABLE-PREFIX, MODULES, +dnl [DESCRIPTION], [DEFAULT]) +dnl ----------------------------------------------- +dnl +dnl Convenience macro to trigger AM_CONDITIONAL after PKG_WITH_MODULES +dnl check._[VARIABLE-PREFIX] is exported as make variable. +AC_DEFUN([PKG_HAVE_WITH_MODULES], +[ +PKG_WITH_MODULES([$1],[$2],,,[$3],[$4]) + +AM_CONDITIONAL([HAVE_][$1], + [test "$AS_TR_SH([with_]m4_tolower([$1]))" = "yes"]) +])dnl PKG_HAVE_WITH_MODULES + +dnl PKG_HAVE_DEFINE_WITH_MODULES(VARIABLE-PREFIX, MODULES, +dnl [DESCRIPTION], [DEFAULT]) +dnl ------------------------------------------------------ +dnl +dnl Convenience macro to run AM_CONDITIONAL and AC_DEFINE after +dnl PKG_WITH_MODULES check. HAVE_[VARIABLE-PREFIX] is exported as make +dnl and preprocessor variable. +AC_DEFUN([PKG_HAVE_DEFINE_WITH_MODULES], +[ +PKG_HAVE_WITH_MODULES([$1],[$2],[$3],[$4]) + +AS_IF([test "$AS_TR_SH([with_]m4_tolower([$1]))" = "yes"], + [AC_DEFINE([HAVE_][$1], 1, [Enable ]m4_tolower([$1])[ support])]) +])dnl PKG_HAVE_DEFINE_WITH_MODULES + m4_include([m4/ax_c_float_words_bigendian.m4]) m4_include([m4/ax_check_openssl.m4]) diff --git a/configure b/configure index cacf9fc41894..b606fc808c17 100755 --- a/configure +++ b/configure @@ -785,7 +785,6 @@ infodir docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir @@ -898,7 +897,6 @@ datadir='${datarootdir}' sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' @@ -1151,15 +1149,6 @@ do | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ @@ -1297,7 +1286,7 @@ fi for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. @@ -1450,7 +1439,6 @@ Fine tuning of the installation directories: --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] @@ -11476,7 +11464,8 @@ fi # checks for library functions for ac_func in alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ - clock confstr ctermid dup3 execv explicit_bzero explicit_memset faccessat fchmod fchmodat fchown fchownat \ + clock confstr copy_file_range ctermid dup3 execv explicit_bzero explicit_memset \ + faccessat fchmod fchmodat fchown fchownat \ fexecve fdopendir fork fpathconf fstatat ftime ftruncate futimesat \ futimens futimes gai_strerror getentropy \ getgrgid_r getgrnam_r \ diff --git a/configure.ac b/configure.ac index 1190b37e9f9d..3d589ac25891 100644 --- a/configure.ac +++ b/configure.ac @@ -3520,7 +3520,8 @@ fi # checks for library functions AC_CHECK_FUNCS(alarm accept4 setitimer getitimer bind_textdomain_codeset chown \ - clock confstr ctermid dup3 execv explicit_bzero explicit_memset faccessat fchmod fchmodat fchown fchownat \ + clock confstr copy_file_range ctermid dup3 execv explicit_bzero explicit_memset \ + faccessat fchmod fchmodat fchown fchownat \ fexecve fdopendir fork fpathconf fstatat ftime ftruncate futimesat \ futimens futimes gai_strerror getentropy \ getgrgid_r getgrnam_r \ diff --git a/pyconfig.h.in b/pyconfig.h.in index b9bb3ffa6f69..20cc901e473b 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -148,6 +148,9 @@ /* Define to 1 if you have the `copysign' function. */ #undef HAVE_COPYSIGN +/* Define to 1 if you have the `copy_file_range' function. */ +#undef HAVE_COPY_FILE_RANGE + /* Define to 1 if you have the <crypt.h> header file. */ #undef HAVE_CRYPT_H

1 0

Document changes for PyCode_New regarding PEP570 (GH-13706)
by Pablo Galindo 31 May '19

31 May '19

https://github.com/python/cpython/commit/545a3b8814dbf2a5391e830d69e796fb1a… commit: 545a3b8814dbf2a5391e830d69e796fb1a1d62ec branch: master author: Pablo Galindo <Pablogsal(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2019-05-31T19:33:41+01:00 summary: Document changes for PyCode_New regarding PEP570 (GH-13706) files: M Doc/c-api/code.rst M Doc/whatsnew/3.8.rst diff --git a/Doc/c-api/code.rst b/Doc/c-api/code.rst index e2b0b23335e3..7aa91ee84d2e 100644 --- a/Doc/c-api/code.rst +++ b/Doc/c-api/code.rst @@ -40,6 +40,9 @@ bound into a function. :c:func:`PyCode_New` directly can bind you to a precise Python version since the definition of the bytecode changes often. + .. versionchanged:: 3.8 + An extra parameter is required (*posonlyargcount*) to support :PEP:`570`. + .. audit-event:: code.__new__ "code filename name argcount kwonlyargcount nlocals stacksize flags" .. c:function:: PyCodeObject* PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 98f0c3474f26..76d00938dbec 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1278,6 +1278,9 @@ Changes in the C API (Contributed by Antoine Pitrou in :issue:`32388`.) +* The :c:func:`PyCode_New` has a new parameter in the second position (*posonlyargcount*) + to support :pep:`570`, indicating the number of positional-only arguments. + CPython bytecode changes ------------------------

1 0

bpo-12639: msilib.Directory.start_component() fails if *keyfile* is not None (GH-13688)
by Miss Islington (bot) 31 May '19

31 May '19

https://github.com/python/cpython/commit/49fc57abf5fcf60129e460046d78c9bf20… commit: 49fc57abf5fcf60129e460046d78c9bf20a19931 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-05-31T10:22:14-07:00 summary: bpo-12639: msilib.Directory.start_component() fails if *keyfile* is not None (GH-13688) msilib.Directory.start_component() was passing an extra argument to CAB.gen_id(). (cherry picked from commit c8d5bf6c3fa09b43f6a5ee779d493d251dbcc53c) Co-authored-by: Zackery Spytz <zspytz(a)gmail.com> files: A Misc/NEWS.d/next/Library/2019-05-30-16-16-47.bpo-12639.TQFOR4.rst M Lib/msilib/__init__.py M Lib/test/test_msilib.py diff --git a/Lib/msilib/__init__.py b/Lib/msilib/__init__.py index 8c5251d9baf8..0bc8dd995246 100644 --- a/Lib/msilib/__init__.py +++ b/Lib/msilib/__init__.py @@ -273,7 +273,7 @@ def start_component(self, component = None, feature = None, flags = None, keyfil if AMD64: flags |= 256 if keyfile: - keyid = self.cab.gen_id(self.absolute, keyfile) + keyid = self.cab.gen_id(keyfile) self.keyfiles[keyfile] = keyid else: keyid = None diff --git a/Lib/test/test_msilib.py b/Lib/test/test_msilib.py index 4aa4753fc2a6..265eaea59b5f 100644 --- a/Lib/test/test_msilib.py +++ b/Lib/test/test_msilib.py @@ -83,6 +83,15 @@ def test_get_property_vt_empty(self): db.Close() self.addCleanup(unlink, db_path) + def test_directory_start_component_keyfile(self): + db, db_path = init_database() + self.addCleanup(db.Close) + feature = msilib.Feature(db, 0, 'Feature', 'A feature', 'Python') + cab = msilib.CAB('CAB') + dir = msilib.Directory(db, cab, None, TESTFN, 'TARGETDIR', + 'SourceDir', 0) + dir.start_component(None, feature, None, 'keyfile') + class Test_make_id(unittest.TestCase): #http://msdn.microsoft.com/en-us/library/aa369212(v=vs.85).aspx diff --git a/Misc/NEWS.d/next/Library/2019-05-30-16-16-47.bpo-12639.TQFOR4.rst b/Misc/NEWS.d/next/Library/2019-05-30-16-16-47.bpo-12639.TQFOR4.rst new file mode 100644 index 000000000000..aade9121b4bb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-05-30-16-16-47.bpo-12639.TQFOR4.rst @@ -0,0 +1,2 @@ +:meth:`msilib.Directory.start_component()` no longer fails if *keyfile* is +not ``None``.

1 0

bpo-25735: math.factorial doc should mention integer return type (GH-6420)
by Miss Islington (bot) 31 May '19

31 May '19

https://github.com/python/cpython/commit/fc3b8437c86167983cc75e5a9a3ed1dc54… commit: fc3b8437c86167983cc75e5a9a3ed1dc542a1b79 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-05-31T09:58:27-07:00 summary: bpo-25735: math.factorial doc should mention integer return type (GH-6420) (cherry picked from commit 4612671df2742eade8ecf8003a6ce1247973c135) Co-authored-by: Akshay Sharma <akshay.sharma09695(a)gmail.com> files: A Misc/NEWS.d/next/Documentation/2018-04-08-19-09-22.bpo-25735.idVQBD.rst M Doc/library/math.rst diff --git a/Doc/library/math.rst b/Doc/library/math.rst index b06b054d516a..fd96d7a95864 100644 --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -50,7 +50,7 @@ Number-theoretic and representation functions .. function:: factorial(x) - Return *x* factorial. Raises :exc:`ValueError` if *x* is not integral or + Return *x* factorial as an integer. Raises :exc:`ValueError` if *x* is not integral or is negative. diff --git a/Misc/NEWS.d/next/Documentation/2018-04-08-19-09-22.bpo-25735.idVQBD.rst b/Misc/NEWS.d/next/Documentation/2018-04-08-19-09-22.bpo-25735.idVQBD.rst new file mode 100644 index 000000000000..8d22cf69e170 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-04-08-19-09-22.bpo-25735.idVQBD.rst @@ -0,0 +1 @@ +Added documentation for func factorial to indicate that returns integer values

1 0