Mailman 3 November 2018 - Python-checkins

Correct grammar mistakes in string.rst. (GH-9752)
by Serhiy Storchaka 07 Nov '18

07 Nov '18

https://github.com/python/cpython/commit/d64991031e4c86ce0331caac1677075751… commit: d64991031e4c86ce0331caac16770757511dd025 branch: master author: Andrés Delfino <adelfino(a)gmail.com> committer: Serhiy Storchaka <storchaka(a)gmail.com> date: 2018-11-07T19:24:56+02:00 summary: Correct grammar mistakes in string.rst. (GH-9752) files: M Doc/library/string.rst diff --git a/Doc/library/string.rst b/Doc/library/string.rst index d0ef089a41a6..46b2bfc82b73 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -551,7 +551,7 @@ addition of the ``{}`` and with ``:`` used instead of ``%``. For example, ``'%03.2f'`` can be translated to ``'{:03.2f}'``. The new format syntax also supports new and different options, shown in the -follow examples. +following examples. Accessing arguments by position:: @@ -739,7 +739,7 @@ these rules. The methods of :class:`Template` are: simply return ``$`` instead of raising :exc:`ValueError`. While other exceptions may still occur, this method is called "safe" - because substitutions always tries to return a usable string instead of + because it always tries to return a usable string instead of raising an exception. In another sense, :meth:`safe_substitute` may be anything other than safe, since it will silently ignore malformed templates containing dangling delimiters, unmatched braces, or

1 0

Mark len call as a code snippet in stdtypes.rst. (GH-9804)
by Serhiy Storchaka 07 Nov '18

07 Nov '18

https://github.com/python/cpython/commit/ca03f3b93ee5c2943a2b8cbf9447f99f83… commit: ca03f3b93ee5c2943a2b8cbf9447f99f835ec672 branch: master author: Andrés Delfino <adelfino(a)gmail.com> committer: Serhiy Storchaka <storchaka(a)gmail.com> date: 2018-11-07T19:22:47+02:00 summary: Mark len call as a code snippet in stdtypes.rst. (GH-9804) files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 49d433805a32..18cd0b06885a 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -3809,7 +3809,7 @@ copying. ``nbytes == product(shape) * itemsize == len(m.tobytes())``. This is the amount of space in bytes that the array would use in a contiguous - representation. It is not necessarily equal to len(m):: + representation. It is not necessarily equal to ``len(m)``:: >>> import array >>> a = array.array('i', [1,2,3,4,5])

1 0

Mark -c and -O as command line options in reStructuredText. (GH-10103)
by Serhiy Storchaka 07 Nov '18

07 Nov '18

https://github.com/python/cpython/commit/ea6a28c9f7e4baa5fe775cebce697a14a7… commit: ea6a28c9f7e4baa5fe775cebce697a14a7d7da8b branch: master author: Andrés Delfino <adelfino(a)gmail.com> committer: Serhiy Storchaka <storchaka(a)gmail.com> date: 2018-11-07T19:06:45+02:00 summary: Mark -c and -O as command line options in reStructuredText. (GH-10103) files: M Doc/reference/executionmodel.rst M Doc/reference/import.rst M Doc/reference/simple_stmts.rst diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 5c83181440bc..1a69e972f2cb 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -22,7 +22,7 @@ The following are blocks: a module, a function body, and a class definition. Each command typed interactively is a block. A script file (a file given as standard input to the interpreter or specified as a command line argument to the interpreter) is a code block. A script command (a command specified on the -interpreter command line with the '**-c**' option) is a code block. The string +interpreter command line with the :option:`-c` option) is a code block. The string argument passed to the built-in functions :func:`eval` and :func:`exec` is a code block. diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst index 7c4f275a4342..d36d7d6a707e 100644 --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@ -951,7 +951,7 @@ In :ref:`the remaining cases <using-on-interface-options>` :mod:`__main__` does not correspond directly with an importable module: - interactive prompt -- -c switch +- :option:`-c` option - running from stdin - running directly from a source or bytecode file diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 1fe1cde1d8d2..5198c0408ce2 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -402,7 +402,7 @@ The extended form, ``assert expression1, expression2``, is equivalent to :: These equivalences assume that :const:`__debug__` and :exc:`AssertionError` refer to the built-in variables with those names. In the current implementation, the built-in variable :const:`__debug__` is ``True`` under normal circumstances, -``False`` when optimization is requested (command line option -O). The current +``False`` when optimization is requested (command line option :option:`-O`). The current code generator emits no code for an assert statement when optimization is requested at compile time. Note that it is unnecessary to include the source code for the expression that failed in the error message; it will be displayed

1 0

bpo-32409: Fix regression in activate.bat on international Windows (GH-10295)
by Vinay Sajip 07 Nov '18

07 Nov '18

https://github.com/python/cpython/commit/c64583b6d3e8516a8cd2b5f84fc1e300bf… commit: c64583b6d3e8516a8cd2b5f84fc1e300bfac2206 branch: master author: samstagern <30337691+samstagern(a)users.noreply.github.com> committer: Vinay Sajip <vinay_sajip(a)yahoo.co.uk> date: 2018-11-07T16:49:14Z summary: bpo-32409: Fix regression in activate.bat on international Windows (GH-10295) Handle Unicode contents on localised Windows systems when activating a venv. activate.bat currently breaks on German Windows systems, as chcp.com does not return a plain number as on English systems, but (arbitrarily) appends a dot at the end (for example "Aktive Codepage: 850." instead of "Active Codepage: 850"). The dependency to chcp.com is removed and ctypes is used to get, set and restore the console output code page. The code page for console input is not changed. We can't use __VENV_PYTHON__ to find python.exe, since it's UTF-8. cmd.exe decodes the script using the console output code page. files: A Misc/NEWS.d/next/Library/2018-11-02-12-01-00.bpo-32409.MFRX2Q.rst M Lib/venv/scripts/nt/activate.bat diff --git a/Lib/venv/scripts/nt/activate.bat b/Lib/venv/scripts/nt/activate.bat index 126049f495fe..2c81a28ce938 100644 --- a/Lib/venv/scripts/nt/activate.bat +++ b/Lib/venv/scripts/nt/activate.bat @@ -1,11 +1,10 @@ @echo off -rem This file is UTF-8 encoded, so we need to update the current code page while executing it -for /f "tokens=2 delims=:" %%a in ('"%SystemRoot%\System32\chcp.com"') do ( - set "_OLD_CODEPAGE=%%a" -) +rem This file is UTF-8 encoded, so we need to update the current code page while executing it. +for /f %%a in ('%~dp0python.exe -Ic "import ctypes; print(ctypes.windll.kernel32.GetConsoleOutputCP())"') do (set "_OLD_CODEPAGE=%%a") + if defined _OLD_CODEPAGE ( - "%SystemRoot%\System32\chcp.com" 65001 > nul + %~dp0python.exe -Ic "import ctypes; ctypes.windll.kernel32.SetConsoleOutputCP(65001)" ) set "VIRTUAL_ENV=__VENV_DIR__" @@ -40,6 +39,6 @@ set "PATH=%VIRTUAL_ENV%\__VENV_BIN_NAME__;%PATH%" :END if defined _OLD_CODEPAGE ( - "%SystemRoot%\System32\chcp.com" %_OLD_CODEPAGE% > nul + %~dp0python.exe -Ic "import ctypes; ctypes.windll.kernel32.SetConsoleOutputCP(%_OLD_CODEPAGE%)" set "_OLD_CODEPAGE=" ) diff --git a/Misc/NEWS.d/next/Library/2018-11-02-12-01-00.bpo-32409.MFRX2Q.rst b/Misc/NEWS.d/next/Library/2018-11-02-12-01-00.bpo-32409.MFRX2Q.rst new file mode 100644 index 000000000000..9b57c5b89ae5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-11-02-12-01-00.bpo-32409.MFRX2Q.rst @@ -0,0 +1,2 @@ +Fixed implementation of :file:`activate.bat` to handle Unicode contents on +localized Windows systems (eg. German). Patch by Martin Bijl. \ No newline at end of file

1 0

bpo-2504: Add pgettext() and variants to gettext. (GH-7253)
by Serhiy Storchaka 07 Nov '18

07 Nov '18

https://github.com/python/cpython/commit/637a33b99685fd5d1032670fbe29c7c8a8… commit: 637a33b99685fd5d1032670fbe29c7c8a8f0ff63 branch: master author: Cheryl Sabella <cheryl.sabella(a)gmail.com> committer: Serhiy Storchaka <storchaka(a)gmail.com> date: 2018-11-07T16:12:20+02:00 summary: bpo-2504: Add pgettext() and variants to gettext. (GH-7253) files: A Misc/NEWS.d/next/Library/2018-05-30-16-00-06.bpo-2504.BynUvU.rst M Doc/library/gettext.rst M Doc/whatsnew/3.8.rst M Lib/gettext.py M Lib/test/test_gettext.py M Misc/ACKS M Tools/i18n/msgfmt.py diff --git a/Doc/library/gettext.rst b/Doc/library/gettext.rst index 38515ebdf591..7f4eab5843f5 100644 --- a/Doc/library/gettext.rst +++ b/Doc/library/gettext.rst @@ -96,6 +96,18 @@ class-based API instead. Like :func:`ngettext`, but look the message up in the specified *domain*. +.. function:: pgettext(context, message) +.. function:: dpgettext(domain, context, message) +.. function:: npgettext(context, singular, plural, n) +.. function:: dnpgettext(domain, context, singular, plural, n) + + Similar to the corresponding functions without the ``p`` in the prefix (that + is, :func:`gettext`, :func:`dgettext`, :func:`ngettext`, :func:`dngettext`), + but the translation is restricted to the given message *context*. + + .. versionadded:: 3.8 + + .. function:: lgettext(message) .. function:: ldgettext(domain, message) .. function:: lngettext(singular, plural, n) @@ -266,6 +278,22 @@ are the methods of :class:`!NullTranslations`: Overridden in derived classes. + .. method:: pgettext(context, message) + + If a fallback has been set, forward :meth:`pgettext` to the fallback. + Otherwise, return the translated message. Overridden in derived classes. + + .. versionadded:: 3.8 + + + .. method:: npgettext(context, singular, plural, n) + + If a fallback has been set, forward :meth:`npgettext` to the fallback. + Otherwise, return the translated message. Overridden in derived classes. + + .. versionadded:: 3.8 + + .. method:: lgettext(message) .. method:: lngettext(singular, plural, n) @@ -316,7 +344,7 @@ are the methods of :class:`!NullTranslations`: If the *names* parameter is given, it must be a sequence containing the names of functions you want to install in the builtins namespace in addition to :func:`_`. Supported names are ``'gettext'``, ``'ngettext'``, - ``'lgettext'`` and ``'lngettext'``. + ``'pgettext'``, ``'npgettext'``, ``'lgettext'``, and ``'lngettext'``. Note that this is only one way, albeit the most convenient way, to make the :func:`_` function available to your application. Because it affects @@ -331,6 +359,9 @@ are the methods of :class:`!NullTranslations`: This puts :func:`_` only in the module's global namespace and so only affects calls within this module. + .. versionchanged:: 3.8 + Added ``'pgettext'`` and ``'npgettext'``. + The :class:`GNUTranslations` class ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -394,6 +425,31 @@ unexpected, or if other problems occur while reading the file, instantiating a n) % {'num': n} + .. method:: pgettext(context, message) + + Look up the *context* and *message* id in the catalog and return the + corresponding message string, as a Unicode string. If there is no + entry in the catalog for the *message* id and *context*, and a fallback + has been set, the look up is forwarded to the fallback's + :meth:`pgettext` method. Otherwise, the *message* id is returned. + + .. versionadded:: 3.8 + + + .. method:: npgettext(context, singular, plural, n) + + Do a plural-forms lookup of a message id. *singular* is used as the + message id for purposes of lookup in the catalog, while *n* is used to + determine which plural form to use. + + If the message id for *context* is not found in the catalog, and a + fallback is specified, the request is forwarded to the fallback's + :meth:`npgettext` method. Otherwise, when *n* is 1 *singular* is + returned, and *plural* is returned in all other cases. + + .. versionadded:: 3.8 + + .. method:: lgettext(message) .. method:: lngettext(singular, plural, n) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 3bacbab1efb6..7b9a9407f0dc 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -131,6 +131,12 @@ asyncio On Windows, the default event loop is now :class:`~asyncio.ProactorEventLoop`. +gettext +------- + +Added :func:`~gettext.pgettext` and its variants. +(Contributed by Franz Glasner, Éric Araujo, and Cheryl Sabella in :issue:`2504`.) + gzip ---- diff --git a/Lib/gettext.py b/Lib/gettext.py index 920742cabe88..72a313a08562 100644 --- a/Lib/gettext.py +++ b/Lib/gettext.py @@ -57,6 +57,7 @@ 'bind_textdomain_codeset', 'dgettext', 'dngettext', 'gettext', 'lgettext', 'ldgettext', 'ldngettext', 'lngettext', 'ngettext', + 'pgettext', 'dpgettext', 'npgettext', 'dnpgettext', ] _default_localedir = os.path.join(sys.base_prefix, 'share', 'locale') @@ -311,6 +312,19 @@ def lngettext(self, msgid1, msgid2, n): return tmsg.encode(self._output_charset) return tmsg.encode(locale.getpreferredencoding()) + def pgettext(self, context, message): + if self._fallback: + return self._fallback.pgettext(context, message) + return message + + def npgettext(self, context, msgid1, msgid2, n): + if self._fallback: + return self._fallback.npgettext(context, msgid1, msgid2, n) + if n == 1: + return msgid1 + else: + return msgid2 + def info(self): return self._info @@ -332,15 +346,11 @@ def set_output_charset(self, charset): def install(self, names=None): import builtins builtins.__dict__['_'] = self.gettext - if hasattr(names, "__contains__"): - if "gettext" in names: - builtins.__dict__['gettext'] = builtins.__dict__['_'] - if "ngettext" in names: - builtins.__dict__['ngettext'] = self.ngettext - if "lgettext" in names: - builtins.__dict__['lgettext'] = self.lgettext - if "lngettext" in names: - builtins.__dict__['lngettext'] = self.lngettext + if names is not None: + allowed = {'gettext', 'lgettext', 'lngettext', + 'ngettext', 'npgettext', 'pgettext'} + for name in allowed & set(names): + builtins.__dict__[name] = getattr(self, name) class GNUTranslations(NullTranslations): @@ -348,6 +358,10 @@ class GNUTranslations(NullTranslations): LE_MAGIC = 0x950412de BE_MAGIC = 0xde120495 + # The encoding of a msgctxt and a msgid in a .mo file is + # msgctxt + "\x04" + msgid (gettext version >= 0.15) + CONTEXT = "%s\x04%s" + # Acceptable .mo versions VERSIONS = (0, 1) @@ -493,6 +507,29 @@ def ngettext(self, msgid1, msgid2, n): tmsg = msgid2 return tmsg + def pgettext(self, context, message): + ctxt_msg_id = self.CONTEXT % (context, message) + missing = object() + tmsg = self._catalog.get(ctxt_msg_id, missing) + if tmsg is missing: + if self._fallback: + return self._fallback.pgettext(context, message) + return message + return tmsg + + def npgettext(self, context, msgid1, msgid2, n): + ctxt_msg_id = self.CONTEXT % (context, msgid1) + try: + tmsg = self._catalog[ctxt_msg_id, self.plural(n)] + except KeyError: + if self._fallback: + return self._fallback.npgettext(context, msgid1, msgid2, n) + if n == 1: + tmsg = msgid1 + else: + tmsg = msgid2 + return tmsg + # Locate a .mo file using the gettext strategy def find(domain, localedir=None, languages=None, all=False): @@ -672,6 +709,26 @@ def ldngettext(domain, msgid1, msgid2, n): DeprecationWarning) return t.lngettext(msgid1, msgid2, n) + +def dpgettext(domain, context, message): + try: + t = translation(domain, _localedirs.get(domain, None)) + except OSError: + return message + return t.pgettext(context, message) + + +def dnpgettext(domain, context, msgid1, msgid2, n): + try: + t = translation(domain, _localedirs.get(domain, None)) + except OSError: + if n == 1: + return msgid1 + else: + return msgid2 + return t.npgettext(context, msgid1, msgid2, n) + + def gettext(message): return dgettext(_current_domain, message) @@ -696,6 +753,15 @@ def lngettext(msgid1, msgid2, n): DeprecationWarning) return ldngettext(_current_domain, msgid1, msgid2, n) + +def pgettext(context, message): + return dpgettext(_current_domain, context, message) + + +def npgettext(context, msgid1, msgid2, n): + return dnpgettext(_current_domain, context, msgid1, msgid2, n) + + # dcgettext() has been deemed unnecessary and is not implemented. # James Henstridge's Catalog constructor from GNOME gettext. Documented usage diff --git a/Lib/test/test_gettext.py b/Lib/test/test_gettext.py index bbad1028dcd8..8c0250eea1e7 100644 --- a/Lib/test/test_gettext.py +++ b/Lib/test/test_gettext.py @@ -15,23 +15,27 @@ # - Tests should have only one assert. GNU_MO_DATA = b'''\ -3hIElQAAAAAGAAAAHAAAAEwAAAALAAAAfAAAAAAAAACoAAAAFQAAAKkAAAAjAAAAvwAAAKEAAADj -AAAABwAAAIUBAAALAAAAjQEAAEUBAACZAQAAFgAAAN8CAAAeAAAA9gIAAKEAAAAVAwAABQAAALcD -AAAJAAAAvQMAAAEAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAAAABQAAAAYAAAACAAAAAFJh -eW1vbmQgTHV4dXJ5IFlhY2gtdABUaGVyZSBpcyAlcyBmaWxlAFRoZXJlIGFyZSAlcyBmaWxlcwBU -aGlzIG1vZHVsZSBwcm92aWRlcyBpbnRlcm5hdGlvbmFsaXphdGlvbiBhbmQgbG9jYWxpemF0aW9u -CnN1cHBvcnQgZm9yIHlvdXIgUHl0aG9uIHByb2dyYW1zIGJ5IHByb3ZpZGluZyBhbiBpbnRlcmZh -Y2UgdG8gdGhlIEdOVQpnZXR0ZXh0IG1lc3NhZ2UgY2F0YWxvZyBsaWJyYXJ5LgBtdWxsdXNrAG51 -ZGdlIG51ZGdlAFByb2plY3QtSWQtVmVyc2lvbjogMi4wClBPLVJldmlzaW9uLURhdGU6IDIwMDAt -MDgtMjkgMTI6MTktMDQ6MDAKTGFzdC1UcmFuc2xhdG9yOiBKLiBEYXZpZCBJYsOhw7FleiA8ai1k -YXZpZEBub29zLmZyPgpMYW5ndWFnZS1UZWFtOiBYWCA8cHl0aG9uLWRldkBweXRob24ub3JnPgpN -SU1FLVZlcnNpb246IDEuMApDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9aXNvLTg4 -NTktMQpDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiBub25lCkdlbmVyYXRlZC1CeTogcHlnZXR0 -ZXh0LnB5IDEuMQpQbHVyYWwtRm9ybXM6IG5wbHVyYWxzPTI7IHBsdXJhbD1uIT0xOwoAVGhyb2F0 -d29iYmxlciBNYW5ncm92ZQBIYXkgJXMgZmljaGVybwBIYXkgJXMgZmljaGVyb3MAR3V2ZiB6YnFo -eXIgY2ViaXZxcmYgdmFncmVhbmd2YmFueXZtbmd2YmEgbmFxIHlicG55dm1uZ3ZiYQpmaGNjYmVn -IHNiZSBsYmhlIENsZ3ViYSBjZWJ0ZW56ZiBvbCBjZWJpdnF2YXQgbmEgdmFncmVzbnByIGdiIGd1 -ciBUQUgKdHJnZ3JrZyB6cmZmbnRyIHBuZ255YnQgeXZvZW5lbC4AYmFjb24Ad2luayB3aW5rAA== +3hIElQAAAAAJAAAAHAAAAGQAAAAAAAAArAAAAAAAAACsAAAAFQAAAK0AAAAjAAAAwwAAAKEAAADn +AAAAMAAAAIkBAAAHAAAAugEAABYAAADCAQAAHAAAANkBAAALAAAA9gEAAEIBAAACAgAAFgAAAEUD +AAAeAAAAXAMAAKEAAAB7AwAAMgAAAB0EAAAFAAAAUAQAABsAAABWBAAAIQAAAHIEAAAJAAAAlAQA +AABSYXltb25kIEx1eHVyeSBZYWNoLXQAVGhlcmUgaXMgJXMgZmlsZQBUaGVyZSBhcmUgJXMgZmls +ZXMAVGhpcyBtb2R1bGUgcHJvdmlkZXMgaW50ZXJuYXRpb25hbGl6YXRpb24gYW5kIGxvY2FsaXph +dGlvbgpzdXBwb3J0IGZvciB5b3VyIFB5dGhvbiBwcm9ncmFtcyBieSBwcm92aWRpbmcgYW4gaW50 +ZXJmYWNlIHRvIHRoZSBHTlUKZ2V0dGV4dCBtZXNzYWdlIGNhdGFsb2cgbGlicmFyeS4AV2l0aCBj +b250ZXh0BFRoZXJlIGlzICVzIGZpbGUAVGhlcmUgYXJlICVzIGZpbGVzAG11bGx1c2sAbXkgY29u +dGV4dARudWRnZSBudWRnZQBteSBvdGhlciBjb250ZXh0BG51ZGdlIG51ZGdlAG51ZGdlIG51ZGdl +AFByb2plY3QtSWQtVmVyc2lvbjogMi4wClBPLVJldmlzaW9uLURhdGU6IDIwMDMtMDQtMTEgMTQ6 +MzItMDQwMApMYXN0LVRyYW5zbGF0b3I6IEouIERhdmlkIEliYW5leiA8ai1kYXZpZEBub29zLmZy +PgpMYW5ndWFnZS1UZWFtOiBYWCA8cHl0aG9uLWRldkBweXRob24ub3JnPgpNSU1FLVZlcnNpb246 +IDEuMApDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9aXNvLTg4NTktMQpDb250ZW50 +LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0CkdlbmVyYXRlZC1CeTogcHlnZXR0ZXh0LnB5IDEuMQpQ +bHVyYWwtRm9ybXM6IG5wbHVyYWxzPTI7IHBsdXJhbD1uIT0xOwoAVGhyb2F0d29iYmxlciBNYW5n +cm92ZQBIYXkgJXMgZmljaGVybwBIYXkgJXMgZmljaGVyb3MAR3V2ZiB6YnFoeXIgY2ViaXZxcmYg +dmFncmVhbmd2YmFueXZtbmd2YmEgbmFxIHlicG55dm1uZ3ZiYQpmaGNjYmVnIHNiZSBsYmhlIENs +Z3ViYSBjZWJ0ZW56ZiBvbCBjZWJpdnF2YXQgbmEgdmFncmVzbnByIGdiIGd1ciBUQUgKdHJnZ3Jr +ZyB6cmZmbnRyIHBuZ255YnQgeXZvZW5lbC4ASGF5ICVzIGZpY2hlcm8gKGNvbnRleHQpAEhheSAl +cyBmaWNoZXJvcyAoY29udGV4dCkAYmFjb24Ad2luayB3aW5rIChpbiAibXkgY29udGV4dCIpAHdp +bmsgd2luayAoaW4gIm15IG90aGVyIGNvbnRleHQiKQB3aW5rIHdpbmsA ''' # This data contains an invalid major version number (5) @@ -84,13 +88,13 @@ UMO_DATA = b'''\ -3hIElQAAAAACAAAAHAAAACwAAAAFAAAAPAAAAAAAAABQAAAABAAAAFEAAAAPAQAAVgAAAAQAAABm -AQAAAQAAAAIAAAAAAAAAAAAAAAAAAAAAYWLDngBQcm9qZWN0LUlkLVZlcnNpb246IDIuMApQTy1S -ZXZpc2lvbi1EYXRlOiAyMDAzLTA0LTExIDEyOjQyLTA0MDAKTGFzdC1UcmFuc2xhdG9yOiBCYXJy -eSBBLiBXQXJzYXcgPGJhcnJ5QHB5dGhvbi5vcmc+Ckxhbmd1YWdlLVRlYW06IFhYIDxweXRob24t -ZGV2QHB5dGhvbi5vcmc+Ck1JTUUtVmVyc2lvbjogMS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFp -bjsgY2hhcnNldD11dGYtOApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA3Yml0CkdlbmVyYXRl -ZC1CeTogbWFudWFsbHkKAMKkeXoA +3hIElQAAAAADAAAAHAAAADQAAAAAAAAAAAAAAAAAAABMAAAABAAAAE0AAAAQAAAAUgAAAA8BAABj +AAAABAAAAHMBAAAWAAAAeAEAAABhYsOeAG15Y29udGV4dMOeBGFiw54AUHJvamVjdC1JZC1WZXJz +aW9uOiAyLjAKUE8tUmV2aXNpb24tRGF0ZTogMjAwMy0wNC0xMSAxMjo0Mi0wNDAwCkxhc3QtVHJh +bnNsYXRvcjogQmFycnkgQS4gV0Fyc2F3IDxiYXJyeUBweXRob24ub3JnPgpMYW5ndWFnZS1UZWFt +OiBYWCA8cHl0aG9uLWRldkBweXRob24ub3JnPgpNSU1FLVZlcnNpb246IDEuMApDb250ZW50LVR5 +cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9dXRmLTgKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzog +N2JpdApHZW5lcmF0ZWQtQnk6IG1hbnVhbGx5CgDCpHl6AMKkeXogKGNvbnRleHQgdmVyc2lvbikA ''' MMO_DATA = b'''\ @@ -147,7 +151,7 @@ def setUp(self): GettextBaseTest.setUp(self) self.localedir = os.curdir self.mofile = MOFILE - gettext.install('gettext', self.localedir) + gettext.install('gettext', self.localedir, names=['pgettext']) def test_some_translations(self): eq = self.assertEqual @@ -157,6 +161,13 @@ def test_some_translations(self): eq(_(r'Raymond Luxury Yach-t'), 'Throatwobbler Mangrove') eq(_(r'nudge nudge'), 'wink wink') + def test_some_translations_with_context(self): + eq = self.assertEqual + eq(pgettext('my context', 'nudge nudge'), + 'wink wink (in "my context")') + eq(pgettext('my other context', 'nudge nudge'), + 'wink wink (in "my other context")') + def test_double_quotes(self): eq = self.assertEqual # double quotes @@ -251,6 +262,20 @@ def test_some_translations(self): eq(self._(r'Raymond Luxury Yach-t'), 'Throatwobbler Mangrove') eq(self._(r'nudge nudge'), 'wink wink') + def test_some_translations_with_context(self): + eq = self.assertEqual + eq(gettext.pgettext('my context', 'nudge nudge'), + 'wink wink (in "my context")') + eq(gettext.pgettext('my other context', 'nudge nudge'), + 'wink wink (in "my other context")') + + def test_some_translations_with_context_and_domain(self): + eq = self.assertEqual + eq(gettext.dpgettext('gettext', 'my context', 'nudge nudge'), + 'wink wink (in "my context")') + eq(gettext.dpgettext('gettext', 'my other context', 'nudge nudge'), + 'wink wink (in "my other context")') + def test_double_quotes(self): eq = self.assertEqual # double quotes @@ -298,6 +323,15 @@ def test_plural_forms1(self): x = gettext.ngettext('There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros') + def test_plural_context_forms1(self): + eq = self.assertEqual + x = gettext.npgettext('With context', + 'There is %s file', 'There are %s files', 1) + eq(x, 'Hay %s fichero (context)') + x = gettext.npgettext('With context', + 'There is %s file', 'There are %s files', 2) + eq(x, 'Hay %s ficheros (context)') + def test_plural_forms2(self): eq = self.assertEqual with open(self.mofile, 'rb') as fp: @@ -307,6 +341,17 @@ def test_plural_forms2(self): x = t.ngettext('There is %s file', 'There are %s files', 2) eq(x, 'Hay %s ficheros') + def test_plural_context_forms2(self): + eq = self.assertEqual + with open(self.mofile, 'rb') as fp: + t = gettext.GNUTranslations(fp) + x = t.npgettext('With context', + 'There is %s file', 'There are %s files', 1) + eq(x, 'Hay %s fichero (context)') + x = t.npgettext('With context', + 'There is %s file', 'There are %s files', 2) + eq(x, 'Hay %s ficheros (context)') + # Examples from http://www.gnu.org/software/gettext/manual/gettext.html def test_ja(self): @@ -646,6 +691,7 @@ def setUp(self): with open(UMOFILE, 'rb') as fp: self.t = gettext.GNUTranslations(fp) self._ = self.t.gettext + self.pgettext = self.t.pgettext def test_unicode_msgid(self): self.assertIsInstance(self._(''), str) @@ -653,6 +699,53 @@ def test_unicode_msgid(self): def test_unicode_msgstr(self): self.assertEqual(self._('ab\xde'), '\xa4yz') + def test_unicode_context_msgstr(self): + t = self.pgettext('mycontext\xde', 'ab\xde') + self.assertTrue(isinstance(t, str)) + self.assertEqual(t, '\xa4yz (context version)') + + +class UnicodeTranslationsPluralTest(GettextBaseTest): + def setUp(self): + GettextBaseTest.setUp(self) + with open(MOFILE, 'rb') as fp: + self.t = gettext.GNUTranslations(fp) + self.ngettext = self.t.ngettext + self.npgettext = self.t.npgettext + + def test_unicode_msgid(self): + unless = self.assertTrue + unless(isinstance(self.ngettext('', '', 1), str)) + unless(isinstance(self.ngettext('', '', 2), str)) + + def test_unicode_context_msgid(self): + unless = self.assertTrue + unless(isinstance(self.npgettext('', '', '', 1), str)) + unless(isinstance(self.npgettext('', '', '', 2), str)) + + def test_unicode_msgstr(self): + eq = self.assertEqual + unless = self.assertTrue + t = self.ngettext("There is %s file", "There are %s files", 1) + unless(isinstance(t, str)) + eq(t, "Hay %s fichero") + unless(isinstance(t, str)) + t = self.ngettext("There is %s file", "There are %s files", 5) + unless(isinstance(t, str)) + eq(t, "Hay %s ficheros") + + def test_unicode_msgstr_with_context(self): + eq = self.assertEqual + unless = self.assertTrue + t = self.npgettext("With context", + "There is %s file", "There are %s files", 1) + unless(isinstance(t, str)) + eq(t, "Hay %s fichero (context)") + t = self.npgettext("With context", + "There is %s file", "There are %s files", 5) + unless(isinstance(t, str)) + eq(t, "Hay %s ficheros (context)") + class WeirdMetadataTest(GettextBaseTest): def setUp(self): @@ -750,6 +843,14 @@ def test__all__(self): msgid "nudge nudge" msgstr "wink wink" +msgctxt "my context" +msgid "nudge nudge" +msgstr "wink wink (in \"my context\")" + +msgctxt "my other context" +msgid "nudge nudge" +msgstr "wink wink (in \"my other context\")" + #: test_gettext.py:16 test_gettext.py:22 test_gettext.py:28 test_gettext.py:34 #: test_gettext.py:77 test_gettext.py:83 test_gettext.py:89 test_gettext.py:95 msgid "albatross" @@ -782,6 +883,14 @@ def test__all__(self): msgid_plural "There are %s files" msgstr[0] "Hay %s fichero" msgstr[1] "Hay %s ficheros" + +# Manually added, as neither pygettext nor xgettext support plural forms +# and context in Python. +msgctxt "With context" +msgid "There is %s file" +msgid_plural "There are %s files" +msgstr[0] "Hay %s fichero (context)" +msgstr[1] "Hay %s ficheros (context)" ''' # Here's the second example po file example, used to generate the UMO_DATA @@ -806,6 +915,11 @@ def test__all__(self): #: nofile:0 msgid "ab\xc3\x9e" msgstr "\xc2\xa4yz" + +#: nofile:1 +msgctxt "mycontext\xc3\x9e" +msgid "ab\xc3\x9e" +msgstr "\xc2\xa4yz (context version)" ''' # Here's the third example po file, used to generate MMO_DATA diff --git a/Misc/ACKS b/Misc/ACKS index aba60945a1e2..3e5fa0a2facc 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -559,6 +559,7 @@ Julian Gindi Yannick Gingras Neil Girdhar Matt Giuca +Franz Glasner Wim Glenn Michael Goderbauer Karan Goel diff --git a/Misc/NEWS.d/next/Library/2018-05-30-16-00-06.bpo-2504.BynUvU.rst b/Misc/NEWS.d/next/Library/2018-05-30-16-00-06.bpo-2504.BynUvU.rst new file mode 100644 index 000000000000..72b0f7044720 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-05-30-16-00-06.bpo-2504.BynUvU.rst @@ -0,0 +1 @@ +Add gettext.pgettext() and variants. diff --git a/Tools/i18n/msgfmt.py b/Tools/i18n/msgfmt.py index 63d52d1f46e5..3f731e941eaf 100755 --- a/Tools/i18n/msgfmt.py +++ b/Tools/i18n/msgfmt.py @@ -5,7 +5,8 @@ This program converts a textual Uniforum-style message catalog (.po file) into a binary GNU catalog (.mo file). This is essentially the same function as the -GNU msgfmt program, however, it is a simpler implementation. +GNU msgfmt program, however, it is a simpler implementation. Currently it +does not handle plural forms but it does handle message contexts. Usage: msgfmt.py [OPTIONS] filename.po @@ -32,12 +33,11 @@ import array from email.parser import HeaderParser -__version__ = "1.1" +__version__ = "1.2" MESSAGES = {} - def usage(code, msg=''): print(__doc__, file=sys.stderr) if msg: @@ -45,15 +45,16 @@ def usage(code, msg=''): sys.exit(code) - -def add(id, str, fuzzy): +def add(ctxt, id, str, fuzzy): "Add a non-fuzzy translation to the dictionary." global MESSAGES if not fuzzy and str: - MESSAGES[id] = str + if ctxt is None: + MESSAGES[id] = str + else: + MESSAGES[b"%b\x04%b" % (ctxt, id)] = str - def generate(): "Return the generated output." global MESSAGES @@ -95,10 +96,10 @@ def generate(): return output - def make(filename, outfile): ID = 1 STR = 2 + CTXT = 3 # Compute .mo name from .po name and arguments if filename.endswith('.po'): @@ -115,7 +116,7 @@ def make(filename, outfile): print(msg, file=sys.stderr) sys.exit(1) - section = None + section = msgctxt = None fuzzy = 0 # Start off assuming Latin-1, so everything decodes without failure, @@ -129,8 +130,8 @@ def make(filename, outfile): lno += 1 # If we get a comment line after a msgstr, this is a new entry if l[0] == '#' and section == STR: - add(msgid, msgstr, fuzzy) - section = None + add(msgctxt, msgid, msgstr, fuzzy) + section = msgctxt = None fuzzy = 0 # Record a fuzzy mark if l[:2] == '#,' and 'fuzzy' in l: @@ -138,10 +139,16 @@ def make(filename, outfile): # Skip comments if l[0] == '#': continue - # Now we are in a msgid section, output previous section - if l.startswith('msgid') and not l.startswith('msgid_plural'): + # Now we are in a msgid or msgctxt section, output previous section + if l.startswith('msgctxt'): + if section == STR: + add(msgctxt, msgid, msgstr, fuzzy) + section = CTXT + l = l[7:] + msgctxt = b'' + elif l.startswith('msgid') and not l.startswith('msgid_plural'): if section == STR: - add(msgid, msgstr, fuzzy) + add(msgctxt, msgid, msgstr, fuzzy) if not msgid: # See whether there is an encoding declaration p = HeaderParser() @@ -183,7 +190,9 @@ def make(filename, outfile): if not l: continue l = ast.literal_eval(l) - if section == ID: + if section == CTXT: + msgctxt += l.encode(encoding) + elif section == ID: msgid += l.encode(encoding) elif section == STR: msgstr += l.encode(encoding) @@ -194,7 +203,7 @@ def make(filename, outfile): sys.exit(1) # Add last entry if section == STR: - add(msgid, msgstr, fuzzy) + add(msgctxt, msgid, msgstr, fuzzy) # Compute output output = generate() @@ -206,7 +215,6 @@ def make(filename, outfile): print(msg, file=sys.stderr) - def main(): try: opts, args = getopt.getopt(sys.argv[1:], 'hVo:',

1 0

bpo-34160: Preserve order of attributes in minidom. (GH-10219)
by Serhiy Storchaka 07 Nov '18

07 Nov '18

https://github.com/python/cpython/commit/5598cc90c745dab827e55fadded42dbe85… commit: 5598cc90c745dab827e55fadded42dbe85e31d33 branch: master author: Diego Rojas <rojastorrado(a)gmail.com> committer: Serhiy Storchaka <storchaka(a)gmail.com> date: 2018-11-07T16:09:04+02:00 summary: bpo-34160: Preserve order of attributes in minidom. (GH-10219) files: M Doc/library/xml.dom.minidom.rst M Lib/test/test_minidom.py M Lib/xml/dom/minidom.py M Misc/NEWS.d/next/Library/2018-10-27-21-11-42.bpo-34160.UzyPZf.rst diff --git a/Doc/library/xml.dom.minidom.rst b/Doc/library/xml.dom.minidom.rst index 40470e8736e7..a1f334d4178e 100644 --- a/Doc/library/xml.dom.minidom.rst +++ b/Doc/library/xml.dom.minidom.rst @@ -143,6 +143,9 @@ module documentation. This section lists the differences between the API and For the :class:`Document` node, an additional keyword argument *encoding* can be used to specify the encoding field of the XML header. + .. versionchanged:: 3.8 + The :meth:`writexml` method now preserves the attribute order specified + by the user. .. method:: Node.toxml(encoding=None) @@ -156,6 +159,10 @@ module documentation. This section lists the differences between the API and encoding. Encoding this string in an encoding other than UTF-8 is likely incorrect, since UTF-8 is the default encoding of XML. + .. versionchanged:: 3.8 + The :meth:`toxml` method now preserves the attribute order specified + by the user. + .. method:: Node.toprettyxml(indent="", newl="", encoding="") Return a pretty-printed version of the document. *indent* specifies the @@ -165,6 +172,10 @@ module documentation. This section lists the differences between the API and The *encoding* argument behaves like the corresponding argument of :meth:`toxml`. + .. versionchanged:: 3.8 + The :meth:`toprettyxml` method now preserves the attribute order specified + by the user. + .. _dom-example: diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py index e91cdba1eb2e..ad5be2f0f0e2 100644 --- a/Lib/test/test_minidom.py +++ b/Lib/test/test_minidom.py @@ -2,6 +2,8 @@ import copy import pickle +import io +import contextlib from test.support import findfile import unittest @@ -1560,5 +1562,24 @@ def testProcessingInstructionNameError(self): pi = doc.createProcessingInstruction("y", "z") pi.nodeValue = "crash" + def test_minidom_attribute_order(self): + xml_str = '<?xml version="1.0" ?><curriculum status="public" company="example"/>' + doc = parseString(xml_str) + output = io.StringIO() + doc.writexml(output) + self.assertEqual(output.getvalue(), xml_str) + + def test_toxml_with_attributes_ordered(self): + xml_str = '<?xml version="1.0" ?><curriculum status="public" company="example"/>' + doc = parseString(xml_str) + self.assertEqual(doc.toxml(), xml_str) + + def test_toprettyxml_with_attributes_ordered(self): + xml_str = '<?xml version="1.0" ?><curriculum status="public" company="example"/>' + doc = parseString(xml_str) + self.assertEqual(doc.toprettyxml(), + '<?xml version="1.0" ?>\n' + '<curriculum status="public" company="example"/>\n') + if __name__ == "__main__": unittest.main() diff --git a/Lib/xml/dom/minidom.py b/Lib/xml/dom/minidom.py index e44e04a069ec..469c51735e03 100644 --- a/Lib/xml/dom/minidom.py +++ b/Lib/xml/dom/minidom.py @@ -854,9 +854,8 @@ def writexml(self, writer, indent="", addindent="", newl=""): writer.write(indent+"<" + self.tagName) attrs = self._get_attributes() - a_names = sorted(attrs.keys()) - for a_name in a_names: + for a_name in attrs.keys(): writer.write(" %s=\"" % a_name) _write_data(writer, attrs[a_name].value) writer.write("\"") diff --git a/Misc/NEWS.d/next/Library/2018-10-27-21-11-42.bpo-34160.UzyPZf.rst b/Misc/NEWS.d/next/Library/2018-10-27-21-11-42.bpo-34160.UzyPZf.rst index 6f3c076d3c03..775d33aacda7 100644 --- a/Misc/NEWS.d/next/Library/2018-10-27-21-11-42.bpo-34160.UzyPZf.rst +++ b/Misc/NEWS.d/next/Library/2018-10-27-21-11-42.bpo-34160.UzyPZf.rst @@ -1 +1 @@ -ElementTree now preserves the attribute order specified by the user. +ElementTree and minidom now preserve the attribute order specified by the user.

1 0

bpo-31553: add --json-lines option to json.tool (#10051)
by Serhiy Storchaka 07 Nov '18

07 Nov '18

https://github.com/python/cpython/commit/f19447994983902aa88362d8fffe645f1e… commit: f19447994983902aa88362d8fffe645f1ea2f2aa branch: master author: HongWeipeng <961365124(a)qq.com> committer: Serhiy Storchaka <storchaka(a)gmail.com> date: 2018-11-07T12:09:32+02:00 summary: bpo-31553: add --json-lines option to json.tool (#10051) * add jsonlines option to json.tool * code review * fix:avoid read infile after it close * improve doc in whatsnew 3.8 files: A Misc/NEWS.d/next/Library/2018-10-23-14-46-47.bpo-31553.JxRkAW.rst M Doc/library/json.rst M Doc/whatsnew/3.8.rst M Lib/json/tool.py M Lib/test/test_json/test_tool.py diff --git a/Doc/library/json.rst b/Doc/library/json.rst index 510e30733fed..589e86ca8107 100644 --- a/Doc/library/json.rst +++ b/Doc/library/json.rst @@ -717,6 +717,12 @@ Command line options .. versionadded:: 3.5 +.. cmdoption:: --json-lines + + Parse every input line as separate JSON object. + + .. versionadded:: 3.8 + .. cmdoption:: -h, --help Show the help message. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 51aee1bed835..3bacbab1efb6 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -151,6 +151,12 @@ by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) The changes above have been backported to 3.7 maintenance releases. +json.tool +--------- + +Add option ``--json-lines`` to parse every input line as separate JSON object. +(Contributed by Weipeng Hong in :issue:`31553`.) + os.path ------- diff --git a/Lib/json/tool.py b/Lib/json/tool.py index 5932f4ecded7..1d82bc824236 100644 --- a/Lib/json/tool.py +++ b/Lib/json/tool.py @@ -26,19 +26,25 @@ def main(): help='write the output of infile to outfile') parser.add_argument('--sort-keys', action='store_true', default=False, help='sort the output of dictionaries alphabetically by key') + parser.add_argument('--json-lines', action='store_true', default=False, + help='parse input using the jsonlines format') options = parser.parse_args() infile = options.infile or sys.stdin outfile = options.outfile or sys.stdout sort_keys = options.sort_keys - with infile: + json_lines = options.json_lines + with infile, outfile: try: - obj = json.load(infile) + if json_lines: + objs = (json.loads(line) for line in infile) + else: + objs = (json.load(infile), ) + for obj in objs: + json.dump(obj, outfile, sort_keys=sort_keys, indent=4) + outfile.write('\n') except ValueError as e: raise SystemExit(e) - with outfile: - json.dump(obj, outfile, sort_keys=sort_keys, indent=4) - outfile.write('\n') if __name__ == '__main__': diff --git a/Lib/test/test_json/test_tool.py b/Lib/test/test_json/test_tool.py index 9d93f931ca39..1e95bc79e5d1 100644 --- a/Lib/test/test_json/test_tool.py +++ b/Lib/test/test_json/test_tool.py @@ -60,6 +60,28 @@ class TestTool(unittest.TestCase): ] """) + jsonlines_raw = textwrap.dedent("""\ + {"ingredients":["frog", "water", "chocolate", "glucose"]} + {"ingredients":["chocolate","steel bolts"]} + """) + + jsonlines_expect = textwrap.dedent("""\ + { + "ingredients": [ + "frog", + "water", + "chocolate", + "glucose" + ] + } + { + "ingredients": [ + "chocolate", + "steel bolts" + ] + } + """) + def test_stdin_stdout(self): args = sys.executable, '-m', 'json.tool' with Popen(args, stdin=PIPE, stdout=PIPE, stderr=PIPE) as proc: @@ -92,6 +114,13 @@ def test_infile_outfile(self): self.assertEqual(out, b'') self.assertEqual(err, b'') + def test_jsonlines(self): + args = sys.executable, '-m', 'json.tool', '--json-lines' + with Popen(args, stdin=PIPE, stdout=PIPE, stderr=PIPE) as proc: + out, err = proc.communicate(self.jsonlines_raw.encode()) + self.assertEqual(out.splitlines(), self.jsonlines_expect.encode().splitlines()) + self.assertEqual(err, b'') + def test_help_flag(self): rc, out, err = assert_python_ok('-m', 'json.tool', '-h') self.assertEqual(rc, 0) diff --git a/Misc/NEWS.d/next/Library/2018-10-23-14-46-47.bpo-31553.JxRkAW.rst b/Misc/NEWS.d/next/Library/2018-10-23-14-46-47.bpo-31553.JxRkAW.rst new file mode 100644 index 000000000000..de80e7cd7d11 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-10-23-14-46-47.bpo-31553.JxRkAW.rst @@ -0,0 +1 @@ +Add the --json-lines option to json.tool. Patch by hongweipeng. \ No newline at end of file

1 0

bpo-34898: Add mtime parameter to gzip.compress(). (GH-9704)
by Serhiy Storchaka 07 Nov '18

07 Nov '18

https://github.com/python/cpython/commit/0e7497cb469b003a45a4abe4105b81ef6d… commit: 0e7497cb469b003a45a4abe4105b81ef6d0c4002 branch: master author: guoci <zguoci(a)gmail.com> committer: Serhiy Storchaka <storchaka(a)gmail.com> date: 2018-11-07T11:50:23+02:00 summary: bpo-34898: Add mtime parameter to gzip.compress(). (GH-9704) Without setting mtime, time.time() will be used as the timestamp which will end up in the compressed data and each invocation of the compress() function will vary over time. files: A Misc/NEWS.d/next/Library/2018-10-04-17-23-43.bpo-34898.Wo2PoJ.rst M Doc/library/gzip.rst M Doc/whatsnew/3.8.rst M Lib/gzip.py M Lib/test/test_gzip.py M Misc/ACKS diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst index a57307b0e499..8850a33f4abb 100644 --- a/Doc/library/gzip.rst +++ b/Doc/library/gzip.rst @@ -157,13 +157,15 @@ The module defines the following items: Accepts a :term:`path-like object`. -.. function:: compress(data, compresslevel=9) +.. function:: compress(data, compresslevel=9, *, mtime=None) Compress the *data*, returning a :class:`bytes` object containing - the compressed data. *compresslevel* has the same meaning as in + the compressed data. *compresslevel* and *mtime* have the same meaning as in the :class:`GzipFile` constructor above. .. versionadded:: 3.2 + .. versionchanged:: 3.8 + Added the *mtime* parameter for reproducible output. .. function:: decompress(data) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 2d3a116df9d3..51aee1bed835 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -131,6 +131,13 @@ asyncio On Windows, the default event loop is now :class:`~asyncio.ProactorEventLoop`. +gzip +---- + +Added the *mtime* parameter to :func:`gzip.compress` for reproducible output. +(Contributed by Guo Ci Teo in :issue:`34898`.) + + idlelib and IDLE ---------------- diff --git a/Lib/gzip.py b/Lib/gzip.py index 151ff1405b16..948fec293e23 100644 --- a/Lib/gzip.py +++ b/Lib/gzip.py @@ -520,12 +520,12 @@ def _rewind(self): super()._rewind() self._new_member = True -def compress(data, compresslevel=_COMPRESS_LEVEL_BEST): +def compress(data, compresslevel=_COMPRESS_LEVEL_BEST, *, mtime=None): """Compress data in one shot and return the compressed string. Optional argument is the compression level, in range of 0-9. """ buf = io.BytesIO() - with GzipFile(fileobj=buf, mode='wb', compresslevel=compresslevel) as f: + with GzipFile(fileobj=buf, mode='wb', compresslevel=compresslevel, mtime=mtime) as f: f.write(data) return buf.getvalue() diff --git a/Lib/test/test_gzip.py b/Lib/test/test_gzip.py index 1e8b41f07b3e..2c8f854c6436 100644 --- a/Lib/test/test_gzip.py +++ b/Lib/test/test_gzip.py @@ -499,6 +499,17 @@ def test_compress(self): with gzip.GzipFile(fileobj=io.BytesIO(datac), mode="rb") as f: self.assertEqual(f.read(), data) + def test_compress_mtime(self): + mtime = 123456789 + for data in [data1, data2]: + for args in [(), (1,), (6,), (9,)]: + with self.subTest(data=data, args=args): + datac = gzip.compress(data, *args, mtime=mtime) + self.assertEqual(type(datac), bytes) + with gzip.GzipFile(fileobj=io.BytesIO(datac), mode="rb") as f: + f.read(1) # to set mtime attribute + self.assertEqual(f.mtime, mtime) + def test_decompress(self): for data in (data1, data2): buf = io.BytesIO() diff --git a/Misc/ACKS b/Misc/ACKS index 89fb0c7000b4..aba60945a1e2 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1615,6 +1615,7 @@ Monty Taylor Anatoly Techtonik Martin Teichmann Gustavo Temple +Guo Ci Teo Mikhail Terekhov Victor Terrón Pablo Galindo diff --git a/Misc/NEWS.d/next/Library/2018-10-04-17-23-43.bpo-34898.Wo2PoJ.rst b/Misc/NEWS.d/next/Library/2018-10-04-17-23-43.bpo-34898.Wo2PoJ.rst new file mode 100644 index 000000000000..4c0a061daf9f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-10-04-17-23-43.bpo-34898.Wo2PoJ.rst @@ -0,0 +1,2 @@ +Add `mtime` argument to `gzip.compress` for reproducible output. +Patch by Guo Ci Teo. \ No newline at end of file

1 0

Daily reference leaks (4243df51fe43): sum=-1
by solipsis＠pitrou.net 07 Nov '18

07 Nov '18

results for 4243df51fe43 on branch "default" -------------------------------------------- test_asyncio leaked [0, 3, 0] memory blocks, sum=3 test_collections leaked [-7, 1, 0] memory blocks, sum=-6 test_functools leaked [0, 3, 1] memory blocks, sum=4 test_multiprocessing_forkserver leaked [-2, 1, 1] memory blocks, sum=0 test_multiprocessing_spawn leaked [0, 0, -2] memory blocks, sum=-2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogfXgJNV', '--timeout', '7200']

1 0

Remove duplicate "Reference Guide" in optparse.rst. (GH-10372)
by Serhiy Storchaka 06 Nov '18

06 Nov '18

https://github.com/python/cpython/commit/d2b11af91560eaaeb499f702f4b0f244ec… commit: d2b11af91560eaaeb499f702f4b0f244ec756280 branch: master author: Andrés Delfino <adelfino(a)gmail.com> committer: Serhiy Storchaka <storchaka(a)gmail.com> date: 2018-11-07T07:47:11+02:00 summary: Remove duplicate "Reference Guide" in optparse.rst. (GH-10372) files: M Doc/library/optparse.rst diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index 3afc77bf9f8e..16f1cf008fff 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -415,7 +415,7 @@ Some other actions supported by :mod:`optparse` are: ``"callback"`` call a specified function -These are covered in section :ref:`optparse-reference-guide`, Reference Guide +These are covered in section :ref:`optparse-reference-guide`, and section :ref:`optparse-option-callbacks`.

1 0